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PREFACE 



ollowlng paper Is one of seven required by the 
rfatio^ i\ institute of Cduoatlon (NIC) grant to the Edu- 
^:2atlc ommlsslon of the States for the administration 
th^ National Assessment of Educational Progress 
^AEP) The papers are Intended to raise significant 
q^^$tions about NAEP functions and procedures In order 
to apprise the Assessment Policy Committers of Its tech^ 
nlcal and administrative options as NAEP moves Into a 
second decade of activity. According to the NIE Program 
Announcement, each paper should Include "discussions 
about the significance of a national assessment of edu- 
cational progress for (a) Informing public opinion, (b) 
Influencing educational and other social policies, (c) 
Identifying specific educational and research needs and 
(d) contributing to the goal of Improving education." 
Thus, the papers combine broad theoretical considera- 
tions with very concrete discussions of potential op- 
tions and their trade offs. All the papers are sugges-^ 
tlve, rather than definitive — starting points for dis- 
cussion rather than end points. 

Two fundamental questions underglrd these papers. 
First, If one were to design a national assessment pro- 
gram from scratch to serve the needs of the eighties and 
nineties, what would It look like? What theory would It 
draw upon? What constituents would It serve? How would 
It operate? And second , given that an assessment has 
been operating for 10 years, how do answers to these 
questions match the currant NAEP? How could the current 
NAEP be changed to accommodate needs and technology un- 
forseen when It was begun In the 1960s? As these ques- 
tions weave In and out of the discussions of various 
Issues, It will become clear that the answers one gives 
will probably depend upon one*s assumptions about auch 
things as the way society, education and science work, 
as well as one's familiarity with the theory, politics, 
history and operation of the current National Assess- 
ment. 

There are many ways to Implement the 1979 legisla- 
tion mandating a national assessment. Some are undoubt- 
edly better than others In certain respects; all have 
both advantages and drawbacks, complicating even the 
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simplest comparisons among alternatives, in such a sit- 
uation, It Is Inperativa that advooates for various nod* 
els of a national assessment adopt an open, interdis- 
ciplinary approach to rethinking the Assessment, listen- 
ing carefully to eaoh other and respeotlng professional 
differences of opinion. 

Individuals charged with making policy decisions 
about NACP — members of the Assessment Polioy Committee 
and members of the U.S. Congress — will find that, in 
many Instances, attempts to strengthen the Assessment in 
one regard will weaken it in some other regard. Given a 
certain level of funding, advances in some areas oome at 
the expense of others. The National Assessment can nei- 
ther be all things to all people nor a single, unchang- 
ing enterprise. It will always represent a oorabination 
of various tensions. The question for its future Is not 
how do we eliminate those tensions? Hather it is; How 
do we harness those tensions creatively to keep the As- 
sessment abreast of the tines and oontlnually relevant 
to America's long-term need for information about the 
status of education? 



INTRODUCTION 



Th« Purpose of This Paper 

This paper addresses two questions raised In the 
National institute of Education (NIE) Program Announce- 
ment for the National Assessment of Educational Progress 
(NAEP) grant: (1) «How can NAEP provide useful Informa- 
tion concerning two central Issues of American education 
— educational equity, and standards and levels of lit- 
eracy, particularly with regard to linguistic and ethnic 
minorities?" And (2), «What kinds of data can be col- 
lected and analyses performed to chart changes in liter- 
acy over time and provide clues about the types of pro- 
grama required for Improving literacy?" 

The answers to the questions are easy to state In 
their most general form: There are many waya NAEP can 
provide useful Information about equity and literacy, 
onoe the terms "useful "equity" and "literacy" are 
deflnedi there are many kinds of data NAEP oan collect 
and analyze in various ways to chart changes In literacy 
over time; and, without major changes in the current 
design of NAEP* there are few things NAEP oan do to pro- 
vide more than the most general olues about the effec- 
tiveness of particular literacy programs. Detailing and 
explaining these general answers In terms that generate 
discussion and clarify policy alternatlvea will be the 
major thrust of the paper. 

Several assumptions underglrd this discussion: 

1 . The Program AnRounoement questions conflate 
several kinds and aspects of measurement that 
must be sorted out and understood separately. 
Some people would view a national assessment 
primarily as a descriptive 3tudy, much like the 
U.S. census; some would view It primarily as an 
analytical study permitting diagnostic analy- 
ses, experiments and research similar to tradi- 
tional educational research; setae would view It 
primarily as an Instrument for large-scale edu- 
cational and social program evaluation . These 
visions, are not fully compatible with one an- 
other, each having its own constituents, its 
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own body of theory and mvthodology, its own way 
of defining utility and Its own history of suo- 
oesses and failures. Sone oomblnatlons are, of 
course , possible, but a national assessment 
« with limited funding Is unlikely to satisfy 
everyone'i needs or desires. The Assessment 
Polloy Committee must weigh the many denends 
made upon NAEP for various kinds of Information 
and deolde whloh will be net at the expense of 
other? An even larger polloy body must deolde 
where a netlonal essessnent, onoe defined, fits 
Into the national ploture of research end In- 
formation gathering. 

2. Deflnl1;lons of literacy have changed, histori- 
cally. There are currently many definitions 
and there will doubtless be others In the fu- 
ture. 

3. Equity can be defined In numerous ways, eaoh of 
which hss very different Implications for meas- 
urement and for policy. 

4. The groups believed to be victims of Inequity 
have changed over tine and will change In the 
future. 

5. Largely because they are fluid concepts defined 
differently by different groups and at differ- 
ent times, both literacy and equity have eluded 
reliable long-term measurement. 

6. The exact relationships between education, lit- 
eracy and equity are obscured by complex inter- 
actions between such fsctors as the educational 
system, the economic system, class attitudes, 
racial attitudes and various soclopclltloal 
Ideologies and Institutions. 

These considerations must give pause to anyone con- 
templating long-term, measurement of changes In these 
areas. For example, the first consideration suggests 
that If NAEP Is designed primarily to be descriptive. It 
may be an Inadequate or Inappropriate tool for evalua- 
tion of such things as successful literacy programs. 
The second and third considerations suggest that the 
measuring instrumentCs) be broad end flexible to accom- 
modate a variety of dynamic definitions of both literacy 
and equity. The fourth suggests that the sample being 
assessed should Include a diverse enough population to 
accommodate dynamic definitions of affected groups of 



Americans. The fifth suggests a need for reaXlstlo ex* 
peotatlons and Imagination. And the sixth consideration 
suggests that the neasurement should not be tied solely 
to traditionally defined eduoatlonaX Institutions and 
programs. At the least, It should be designed to Inte* 
grate well into sources of noneduoatlonal data suoh as 
oensus studies, labor statlstlos and the like. 

In the pages that follow, eaoh of these suggestions 
wm be expanded upon with referenoe to the National 
Assessment of EduoatlonaX Progress as It Is presently 
defined and as It might be defined In the future to bet- 
ter meet needs for Information about literacy and eq- 
uity. Part 1 win explore differing oonceptlons of Xlt- 
eraoy, ways In whloh NAEP can address certain of them, 
ways In which NAEP oan serve as a proving ground for 
literacy measures, and potential enhancements to the 
ourrent NAEP model. Part 2 will explore NAEP's oontrl- 
butlon to equity studies as a desorlptlve data base and 
a source of anaXytloal inquiry. Part 3 win examine 
trade offs Involved In potential enhancements of the 
ourrent design. 



A Framework for Weighing Policy 
and Design Alternatives 

Three major tensions permeate all facets of the 
Assessment and influence all deolslons about how It 
might be changed . The first la the tension between 
Short-term and long-term interests i a decision to Im- 
prove NAEP's utility In the short-term could impaot ad- 
versely upon Its long-term mission, and vice versa. The 
seoond Is a tension between breadth/descrlptlveness and 
depth/analytloal oapaolty. And the third Is the tension 
between the Informational needs of diverse clients. 
Many things can be aocompllshed within the current NAEP 
framework that balarioe short- term/long-term, bread th/- 
depth and audience Interests. But, given limited re- 
souroes, these polar consld- —atlona oompete with one 
another. Discussion of NAEP's oontrlbutlons to llteraoy 
and equity knowledge requires a general understanding of 
this competition and Its consequenoes for the Assess- 
ment's design. 

The short-term/long-term trade off Is easy to un- 
derstand. If we tailor an assessment to address today's 
Issues on today's timetable. It may prove useless 10 
years hence. Not only might the Issues have changed but 
the methodology for getherlr:s short-term data oould be 
Inappropriate for long-term measurement. 
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The audienoa trade off problem is equeXXy oXear. A 
'*Xatent trait** approaoh to neaaurlng reading perfornenoe 
might provide information usefuX to psyohoXoglsts but 
IncomprehenslbXe or maddening to reading professloneXs 
Interested In a different kind of date or e Xarger pooX 
of NAEP reading items. Many ways of ooXXapslng, struo- 
turlng, scaXlng or weighting data represent oXerlfloa- 
tlon to some eudlenoes but mystlfloetlon to others. 
Since NAEP reXles upon the voXuntary cooperation of 
schooXs and the good wlXX of various aubjeot-matter pro- 
fesslcnaXs, asaocletlons end poXltloaX groups, audience 
trade offs are not trlvlaX considerations. 

The breadth/depth, descrlptlve/anaXytlcaX trade off 
deserves speclaX comment because It beers espeolaXXy on 
the Issues of Xlteraoy and equity. As It is ourrentXy 
designed , the NatlonaX Assiifssnent represents e hybrid 
descrlptlve/anaXytlcaX underteklng. On the descriptive 
side, breadth la the primary consideration. It empXoya 
a broad sampXe, a breed coverage of subject areas, a 
broad coverage of objectives within subject ereea, many 
different kinds of meaaures, breed background verlebXes 
and an assessment not onXy of knowXedge but of sklXXs 
and attitudes. Breadth requires discrete exercises, 
matrix sampXlng, natlonaX units of aneXysls and the Xo- 
glstlcs and time commitments associated with Xarge-soaXe 
survey work. Descriptive data tend to be Xlke aerlaX 
photographs, reveeXlng features that cennct be seen on 
the ground; they Invite dlsolpXlned observation, ru- 
mination , a thought f uX stud y of f ao ts slmlXar to the 
activity of the historian. Descriptive deta suggest 
hypotheses and point to potentlaXXy frultfuX areas of 
more focused research. They do not Xend themseXves to 
focused decision making; rather, they contribute to an 
atmosphere of beXlefs, ideas and theories. 



On the aneXytloaX side, depth Is the primary con- 
sideration. The hunt for reXetlonshlps between soolo- 
educatlonaX factors and eduoetlonaX performance Is best 
faolXltated by a sampXe deep enough for fine-ceXX anaXy- 
sls and by precise ourrlcuXar, resource and sooloeoo- 
nomlc varlabXes. Some of the features that make NAEP a 
usefuX descriptive study (e.g., its breadth. Its matrix 
sampXe) can be nulaenoes to researchers who are In- 
terested m cross-seotlonaX work, more easlXy ccXXapsl- 
bXe data, poXloy tlmeXlness or experlmentaX contrcX. 
AnaXytlcaX data shouXd bear on speolflo poXloy questions 
and ShouXd invite hypothesis testing and mathematloaX 
modeXlng. if NAEP*s descriptive function is Xlke taking 
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aerial photographs of a dam site, Its analytical fune* 
tlon 19 like conducting environmental impact studies. 
Both are critical to dam building but each Is a distinct 
activity* To a certain degree, NAEP permits both func- 
tions. 

As Figure 1 suggests, a certain amount of analyti- 
cal work has always been done and there Is more to be 
done within the current model. But there are limits to 
NAEP*s analytical capacity; without massive funding, an 
Increase In depth can only be purchased at the expense 
of breadth* One of the Assessment Policy Committee* s 
major tasks In the next decade, then, will be to weigh 
the comparative contributions of NAEP*s descriptive and 
analytical functions and to consider the long-term im- 
plications of a reorientation in one or the other direc- 
tions. 

If the suggestions raised throughout this paper are 
evaluated within a policy framework such as the one 
sketched here, the National Assessment can move into the 
eighties wit^ a thorough knowledge of its capacity to 
respond to ou^ and remain an Important, unique source 
of information auout American education. 
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FIGURE 1. The Breadth/Depth Trade Off 
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PART 1 



LITERACY 



Introduotlon 

It Is not surprising to find literacy and equity 
discussed m the same paper, since the former has long 
been thought to be a precondition for the latter. Ever 
since Horace Msnn defined education as "the balance 
wheel of the social machinery" there have been those who 
felt that a truly equitable society would not evolve 
until all Americans shared a level of literacy enabling 
each to compete for the social goods and rewards he or 
she linked to the pursuit of happiness. As society has 
grown more technical, Increasing numbers of Americans 
have sought more and more education In the belief that 
^higher. 4evels-of Hteracy~are-need ed-to-keeprabreast of 
progress. Klriorltles have looked to education as a pri- 
mary means of remedying social Inequities and acquiring 
a fairer share of the social goods. Until recently, few 
Americans questioned the values of literacy for social 
and economic success or improving the life chances of 
minorities. 

However, a number of developments have challenged 
the conventional wisdom and created uncertainties about 
the meaning of the word literacy, the level of literacy 
required to pass through public schooling cr "survive" 
economically after graduation, and the precise relation- 
ship of literacy to various economic and social out- 
comes. - ~ ■ 

One.. development has been a profound disenchantment 
with the public schools and a widespread belief that, 
whatever literacy la, the schools ere not Inculcating 
enough of It Into American youngsters. Declining test 
scores, particularly In verbal skills, reading end writ- 
ing, .have eontrlbutad to a fear that the present genera- 

>fi\?;_be unequal to the demands an increasingly com-, 
plicated, communications-oriented society will make upon 
.it... . At the same time, paradoxically, numerous social 
thinkers have begun to argue that literacy may not be as 
Important as literate people would like to believe. 
Chamjplons of electronic media, for Instance, have argued 



that traditional^ prltit-orletited literacy skills will 
soon be— passe ^ If they; are not-already (e.g^ McLuhan^ 
1969)-^ Bell Telephone advertisements suggest that let- 
ters are „eostly« Inefficient means of doing business. 
Talking computers, In formatlbri-prooes sing machines and 
satellite hookups are predicted to free us from labor- 
some, **llnear** ways of communicating, such as reading 
and writing. Social critics such Christopher Lasch 
(19T9 ) and Randall Collins (1979) sug^^^^ that advanced 
Industrial society not need a highly literate popun 

latlon because fewer and fewer jobs really require the 
careful thinking associated with reading and writing . 
The best evidence that they are not needed lies In the 
fact, Lasch asserts, that they are not being produced. 

and others, a steady trend In American educational his- 
tory away from standards of high literacy toward trivia- 
lized education for the "great army of Ihcapables.** Such 
a trend relegates literacy to narrower and narrower do- 
mains. 

Le3s pessimistic critics, such as Williams (1958), 
suggest that literacy remains Important but not as crit- 
ical -to -the -quallty-of-^l literate people- 1^^ 
believe.. Other studies corroborate this by providing 
evidence that people overrate the literacy demands of 
jobs and underrate human Ingenuity and resourcefulness. 
Jehcks, for example, notes that students who leave high 
school with 8th grade reading 3kllls are by no means 
unemployable: "At least In econonlc terms ^" he writes, 
"the cost of reading at 8th grade rather than 12th grade 
level Is quite small" (Jencks , 1972 , p. 110). A great 
many people seem to be functioning well who would be 
judged "functionally Illiterate" by-one test-or another. 
In this connection, too, a point made by Daniel 
Boorsteln (1967) may be worth keeping In mind. When 
America had a scarcity of legal and medical learning, 
the result was—not a scarcity of lawyers and doctors, 
but the development. Instead, of new kinds of lawyers 
and doctors, new concepts of law and medicine. 

Another development challenging traditional notions 
about literacy Is a barrage of widely publicized studies 
of the relationship between education and success show- 
ing that education has not markedly closed the gap be- 
tween rich and poor Americans In this century (Blau and 
Duncan, 1978; Sewell, 1971; Jencks, 1972; Bowles and 
Glntls, 1976; Brlttaln, 1977; Dorn, 1979). As some of 
the tables In Appendix A reveal, there have been In- 
creases In educational attainment for all Americans, but 
these have not always or consistently led to decreases 
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in_ the inooma gap batwesn havss and have nota. Thus, 
pollpy makers are oonfronted with some evidence that 
inopeased litapaoy translates into economic advantage 
and some eviderioe that it does not. And even if they 
ohoosa to beli«ve the fopmer, they are also presented 
with oontpadiotopy evidenoe about how schools oca "make 
a diffepanoa.*' 

To add to the oonfusion, e revival of plupalist 
sooial theopy has fostepad the idea that the litepacy 
needs of diffapent gpoups of Amepioans differ with theip 
values, languages, dialeots, tpaditions and goals. A 
oontaxtual, pelatlvistio view of litepacy, defining it 
as an ability to manipulate the symbolic code of one's 
oultupe OP suboultupe, both bpoadens the ooneept beyond 
peading and writing and narpows it to the needs of a 
given individual in a given enviponment with given 
goals. Suoh an apppoaoh bpings sophistication to the 
oonoept but fpustpatas polioy making. 

Eaoh of these developments — confusion about ex- 
isting levels of litapaoy, the efficiency with which 
sohools ape ppomoting litepaoy, the true function of 
litapaoy in today* s and tomoppow*s societies, the pela- 
tion between litepaoy and wealth op quality of life and 
the ppasenoe of many definitions of litepacy fop diffep- 
ant gpoups and situations »» oomplieates a msttep onoe 
viewed as pelatively simple. Concappent with these com- 
plioatlons, howavep, th«re pemains a pepsistent belief 
that we know nope than enough about litepacy to undep- 
stand and pponcte it effaotively. Fop most people lit- 
apaoy means ability to paad and write at a level that 
faoilitates smooth entpy into adulthood and an ability 
to pupsua happiness as an individual may choose to de- 
fine it. At this genepal level we all feel we know what 
litepaoy is, we know the schools ape ppimapily pesponsi- 
ble fop taaohlng It and we know that it is valuable both 
as means to sooial op aoonomio ends op as an end in it- 
self. 



The Situation a national assessment faces, then, is 
this: On the one hand, thepe is widesppead belief that 
we know what litepaoy is, whepe people acquipe it and 
how. it is used; on the othep hand, there is evidence 
that beyond a vepy genepal definition, litepaoy is dif- 
fioult to define to evepyone's satisfaction, litepacy 
levels and standapds ape even mope oontroversial than 
definitions, the oonoept of literacy is changing rapidly 
and there ia iBiioh we do not know about how it is best 
taught and how it translates into particular psychologi- 
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0*1 1 90o.lal or eoonomlo benefits . The political and 
ptapt^lojl implloatlons, of. thls^ mixture of knowledge and 
Ignorance for anyone setting out to neasure literacy 
Over the long tern should be obvious. 

Llterecy Issues HASP Could Address 
Without Mejor Design Changes 

One approach to Identifying future rolea for a ne- 
tlonel assessment is to determine whil^ the present Na- 
tional Assessment can and cennot do. If policy makers 
decide It ehould do more than it does now or do eome- 
thlng different, they can examine poteptlel jjio^lflca- 
tlons. Here we ere concerned with contributions to un- 
derstanding the problem of llterecy thet NAEPt uniquely, 
can meke within ita present theoretical and prebtlcal 
constraints. The word .uniquely must be etressed , for 
there is no point in redesigning NAEP to provide infor- 
mation now available from other sources. Many changes 
could be made within the present model to increase its 
contributions, but these will not be considered major. 
Major changes wOuld involve reconceptuallzlng NAEP*s 
role and redesigning the. entire model} they will: be con- 
sidered In the next section. " 

The current Natlcnal Assessment is well suited to 
eddress questions about the distribution of literacy in 
America and questlona about efficient means of asseaslng 
end monitoring varlcus literacies. 



The Distribution of Llterecles 

Estimates of illiteracy or **functional illiteracy" 
m America range from ^% to 50% of the adult population 
(U.S. Census, 1969, 1977; Harmon, 1970; Harris, 1970j 
Vogt, 1973; Adult Performance Level Project, 1975; Fiinc- 

?ig5*^„>l^*''"/-„r •- ' ^576; Murphy, 1973; CoppeTiarTT 
1978; Flsner, 1978; Harmon and Hunter, 1979). The rea- 
eon there Is so much discrepancy In estimates is thet 
different studies employ different definitions of liter- 
acy, different measuring instruments and different sam- 
ples of the population. If one defines illiterates as 
people who are 14 years old or older but have completed 
fewer than five years of schooling, the figure is 2,8K, 
If one defines illiterates as people who cannot perform 
certain reading and writing tasks deemed "functional" by 
some group of "experts," the figure can be as high as 
50». And If one defines literacy as the ability to read 
demanding materials with good comprehension and write 
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olear prose suitable to partioulap audiences, nost Amep- 
ioans may be in hot water,:; This Babel has impeded ef- 
forts to disooyep the true magnitude of literaoy and 
illiteracy in Amerioa. Fortunately, however, the na- 
tional Assessment has been broad enough to aoooanodate a 
number of definitions ^ of literaoy over the years. 
Should it continue to be defined broadly, it will aooom- 
modate a number of .future definitions es well, permit- 
ting the kind of long-term measurement that has been 
elusive in the past, A brief review of some major def- 
initions of litereoy and NAEP design oharaoteristios 
should serve to illustrate this particular strength of 
the Assessment, 

Those who define' literaoy aa a prooess see it as an 
Intention to make meaning, an attempt bo understand in- 
formation, or a oottplex of largely unoonsoious psyoho- 
logical , cognitive and social activities — most of 
which are beyond the reach of traditional measurement 
tools. Those who define it behaviorally list many lev- 



1, The ability to read and write one's name. 

2, The ability to read such materials as are crit- 
ical to "survival" <i,e., legal documents, 
health and safety information, job notices, 
application forms) and to write sufficiently to 
fill out forms, 

3, The ability to perform reading and writing 
tasks rer^uired for performing one's job satis- 
factorily, 

4, The ability to read with comprehension a range 
of materials for a variety of purposes and to 
write a range of communications for a variety 
of purposes and audiences, 

5, The ability to perform <3) and to perform fun- 
damental mathematical computations and access 
resources such as libraries. 

6, The ability to perform <i|) and to act aggres- 
sively in behalf of one^s rights and responsi- 
bilities as a citizen. 



7, Mastery of fundamental processes of reading, 
writing , problem solving , computing , speaking 
and listening; and mastery of a core knowledge 
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base In the eolenoee and humanities; and raes- 
tery„of basio. to,p.ls. for study. sufficient to 
•neble one to pursue any personel goals m this 
society. 

Lltereoy (1) is usually thought of as too mlnlmel 
for. the nibdern world » Llteraoles (3) end (3) embreoe a 
veriety of "•funotlonal," "•mlnlmel'* and "survival" defi- 
nitions. Llteraoles and (5) enbraoe likely outoomes 
of a high- aohool eduoatloh. Llteraoy (6) esaumea an 

additiimiri^^^^^ 

pasa aa the goal of a liberal education. Eeoh of these 
definitions has a oonstltuenoy with a soolel or aduoe- 
tlonel egenda. end a need for deta. In. addition to 
these, one hears of "solentifid," "oonsumer," "eo- 
onomlQ," "hlstorloel" and" "therapeutlb" "literabiea. 
Presumably, ona oan be liberally adubatad end ; remeln 
Ignoiraht about sueh survival mattera as energy, foreign 
affelrs end infletlon, or aueh quellty-of*llfe miatters 
as^ ar^t, .muslo . end. phllosophy. . . As more end mora Amarl- 
eens are eduoeted out of old definitions of lllltereoy, 
the soolety saana to oreate new definitions to worry 
about. 

Faoed with many deflnltlona and levels of llteraoy 
and the , promise of more to.eome, one who would assasa it 
over the long run nuat have a flexlbla end inoluslve 
Instrument. This the Asseasment la, by., design. Some 
Important relevant features; 

1. Eeeh aaseaament elms to gather informetlon 
about a wide range of oonoerna. The reading 
assessment Involves "survival" tasks, suoh as 
reading parking tickets. Job notices, safety 
Information and the like; but it elso Involves 
reading editorials, graphs, stories, blog- 
rephlea and even poems. The writing assessment 
requires skill In filling out forms and oompos- 
Ing Job applloatlon lettera; but it also es- 
aeaaea abilities to oomposa expressive and per- 
suasive essays, write varloua kinds of letters 
and revise first drefts. The mathemetloa as- 
sessment, ea wall, oovera a range of skills 
from the simple to the complex. If, 20 years 
henoa, mathematloal llteraoy ("numeraoy") is 
defined as an ability to perform algebralo 
taaka, the Assessment will have relevant data 
from the 1970s beoauae It included algebra in 
Ita early assesaments. Had the assessment been 
defined In terms of baalos, it would have 
proven too narrow for long-term use. 
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Should a gensral llt«raoy definition AXpAnd to 
include a smattering of skills aoross a number 
of subjeot areas, a broad HAEP working in at 
least 10 areas oould aobommodate the expansion. 
A more narrowly oonoelved NAEP — limited to 
only reading, writing and miathematlos — would 
be less useful J,n the long run^ In sddltlon, a 
oomprehenislvi NAEP permits iriterdlsolpllnary 
study of lltaraoy — I.e., reading and writing 
m the contexts of science, citizenship, work 
or the arts. Policy makers must carefully as- 
sess the long-term consequences of a narrowly 
conceived NAEP. While such a program may ap- 
pear to be cheaper now, it could be extrava- 
gantly expensive In terms of the amount of dsta 
lost and the cost of chronic retooling to ad- 
dress short-term issues. 

Any single assessment includes, as noted, easy, 
as well as difficult, iaaterlals. The Assess- 
ment Is not tuned to a cutoff point to dlsorlm- 
Inate among Individuals and rank order them. 
In addition to thls„.adyjn^tage, every assessment 
Includes (or should include) a^ysriety of 
Items, Item fprmsts, performance tasks and 
questionnaires. The writing assessment re- 
quires writing (many writing tests do not); the 
reading assessment Includes short and long 
written responses as well as multiple-choice 
questions; the mathemetlcs assessment also In- 
cludes open-ended response tasks. All assess- 
ments gather Information about attitudes as 
well as achievement levels. This diversity or 
approach has obvious advantages over the use of 
an Instrtiment that relies exclusively upon one 
type of Item or format or response' mode. (See 
Appendix c for descriptions of NAEP materials 
m reading, writing end literature.) 

The very process of defining and cresting an 
assessment Is conducive to long-term utility. 
The fewer the groups represented In the 
objectives- and exercise-development phases, 
the fewer the groups who will find their def- 
inition of literacy In the assessment. NAEP 
employs a consensus, approach that Includes the 
concerns of a wide range of groups. Besides 
Its obvious relevance to long-term utility, 
this approach also serves periodically, to bring 
Into focuis a national perspective on literacy 
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that Is loportant in itaaXf. 

1*. Tha . AaWaamant aan orltloaX - aga 

groups. ^ XdViUy, it ahouXd aampXa aduXts as 
waJLX,,^ not. onXy to put- tha rasuXts of tha 
so^lioeX-aga^ youngstars into, parapaotlva, but to 
monitor vdhaiiiias In tha rata* aT wHloK^adurts^^^^^^^ 
: tand to Xo»» or inp.pa^s,a.tl>«ir xitapaby skixxs. 
In addltlenf tha Aasassinfnt aampxis ^aahv pbp^^^ 
.\ftiona :jDf:;^^u?^^^ 

HlspanoSf Out-oNsohoQX 17-yaar-bXda and rurax 
Inhabitants -- aXX groups with h'ighar tlian av- 
araga ratas of IXXltaraoy. Aa tha Assaasmant 
anhanoaa Its oapaolty, to anaXyza thaaa groups* 
abhlavamants by oihar varlabXas iuoh as paran- 
tax aduoatlbhf "sobib'lbio^^ 

nlty and sohooX oharabtarlstlbs, Its ablXlty to 
nbnltor— trands for poXlby-raXarant'-groups wlXX 
Inoraaaa. 

TO IXXustrata how thasa faaturas oontrlbuta to Xlt- 
•r«by anaXysls, oonsld«r tha foXXowlng situation. Two 
gppupa of pabpla, hava arrlvad at two dlffarant daflnl- 
tlons of lltaraoy. and wbuXd Xlka tb knbw how thalr ait-- 
araolaa «ha dl'strlbOtad aaone AaaplQan"^ i7»yAk r»oidg' »hrf 
how thay ara jMnilng. / GrbupLA^ d^^^ 

tha mastary of sona fundainantaX raadlng, writing^ oonpu* 
tation . and ..thlnklng,..sklUs. ..Tha group daflnaa .raadlng 
litaraoy In tarns of "fiinbtlbiVaX / "lltaraX," ^•raaX 
worXd/ "avaryday" raadlng tasks. It daflnas wrlttVig 

Xltaraoy^ .ln. taras of„knowXadga of. grammar, and possasslon 

of Xow-XavaX sklXXs suoh aa tha ablXlty to wrlta a oom- ^ 
pXata santanoa or paragraph oorraotXy, l.a,, with oor- 
raot spaxxingf oapltaXlzatlon and punotuatlon. It ba- 
Xlavaa that high aohbbX graduatas shouXd ba abXa to add, 
subtrsotf muXtlpXy and dlvlda, as waXX as know baslo 
mathanatloaX faots. And, whan daflnlng Xltaraoy In 
tarma of tha oognltlva aklXXs InvoXvad In aoqulrlng It, 
tha group Is intaraatad In tha lowar oognltlva XavaXs of 
knowladga and oomprahanslon. 

Group BfaaXs that hlghar-XavaX sklXls ara raqulrad 
for a maanlngfuX oonoapt of Xltaraoy. Tha group la mora 
Intaraatad In "InfarantlaX" raadlng sklXXs than In Xlt- 
araX oomprahanslon. It daflnas writing sklXX In tarms 
of ablXlty to mastar daap struotura, rathar than aurfaoa 
struotura aspaots of writing, and ablXlty to addrass 
dlffarant audlanoaa and situations with tha approprlata 
strataglas. It oonaldars undarstandlng and appXlbatlon 
of mathamatlos to hava mora Xong-tara baarlng on young- 
stars* llvaa than mastary of Xow-XavaX aklXXs. Aiid, 
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when defining literaoy in ttrms of the cognitive skills 
IV requires,- the group feela that applioation, analysis 
and synthesis are oentral to a literaoy one oan use 
' fruitfully. How muoh oould the groups learn from HAEP 
data? What improvements oould be made to provide them 
with more or better data? 

Literaoy A . If group A browsed through HAEP data in 
reading, writing, mathematios, soianoe, funotional lit- 
eracy, basic math <double sampled in ,1975-iT6) and lit- 
eral comprehension <triple sampled in l97i»-75) looking 
at both aggregated and e:(eroise-level data, here is what 
they would discover <see Appendix A for a more complete 
explanation) ; 

1 . The vast majority of America's 17-year-olds 
appear to have command of very basic reading, 
writing, computing and thinking skills. 

2. Even when percentages are as high as 90% on 
basic tasks, that still leaves lOK of the 17- 
year-olds who may not be able to perform them, 
and that translates into hundreds of thousands 
of people — not an insignifiqant number. 

3- Percentages are much lower for minority groups 
and people in Icw-SES <aocioeccnomic) cate- 
gories. As basic as Literacy A is, dispropor- 
tionate numbers of people in these groups ap- 
pear not to have achieved it. 

4. The Situation does not appear to be worsening. 
The percentsgo of l7-year-clds possessing Lit- 
eracy A is not declining, and some groups are 
narrowing the gap between themselves and the 
nation* 

Literacy- B . The reading component of Literaoy B is 
defined as possession of "higher-level" and "inferenc- 
ingt* skills. Although it sounds simple enough to sepa- 
rate - those kinds of skills from the "lower-level" 
skills, it is not an easy matter. Theorists of reading 
oonprehension can muddy interpretations of test results 
by arguing that both high- and low-level skills are 
probably at work in any aot of oomprehensioh. " Students 
may do poorly on so-called literal comprehension items 
either because they lack some low-level skills <e.g., 
decoding) or; because they lack some higher-level ones 
(e.g., forecasting) . Students may perform less well on 
inferential tasks either because they lack some skills 
peculiar to inferenoing or because they simply have not 
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. ma8tar«d th« lowap akllls yet. Fop purposas of this 
P«P„«E:t..wa.,will aasuma that group B bal lavas hlghar-laval 
ooinprahanslon skill Is mora than tha sum of a number of 
lowar-laval skills. That Is, it Ihvolvas soma obghltlva 
aotlvltlas- that ara different from thbsa engaged In 
low-level teske. Consequently, a policy implloetldn of 
changes In hlgher-leval proQesslng skills- Is thst they 
qehnot be Improyad simply by tesohlng the lower-level 
skills. L16eraoy~ B oennot be Improved simply by teaoh- 

Ve will make the seme essumptlon for writing, math- 
ematloa, applloatlon skills, and analysis and synthesis 
skills. Llterady B Includes some elements of Llteraoy A 
and Is somewhet dependent upon it, but It also repre- 
sents a different kind of activity fostered by different 
teaching and laernlhg models. 

Hare Is whet group B Is likely to flndt 

1. Although the mejorlty of 17-year-olds has ac- 
quired Literacy B, a considerable minority — 
perhaps as many as a third -- has not. This 
<trjih8late8..„lnto .perhaps., a-mll'llon-youngst^rs 
ebttut toentar the Ijbb ma who lack or have 
Inconsistently mastered skills affecting" their 
quality of Ufa and their .opportunities. 

2. The percentage of 17-year-olds In possession of 
Lltereoy B Is declining. 

3. The percentages are even lower for minorities 
and people In lower SES cetegorles. And there 
are no signs that the gap between them end the 
national population Is closing significantly. 

Both groups A and B would be expected to define 
their llterecles carefully; the analysis could be done 
using any set of exercises and variables the groups 
choose. They would be expected to view the results In 
the light of other educetlonal and social information. 
The point worth stressing Is that two conceots cf llter- 
ecy with different policy Impllcetlons could be pursued 
because the assessments , have been broad enough to sup- 
port them, in aaaitlcn., a. group C miahfe have AmfinmA 
literacy m terms of consumerism, and another might have 
defined It in relation to the fine arts. All could find 
complex — which Is to say more precise — data thet do 
not exist snywhere else. No other Indices — SAT 
scores, one-time studies, Gallup polls, etc. — are half 
so rich In Information. 
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How«v«p, aa thlngaourrentXy stand, groups A and B 
would npt find MAEP data easy to work with. The anaXy- 
S9S of Lit«r«oies A and B described in Appendix B were 

dona QUioklV bv anm»nn» f.hnrniis hi v ..4^u uapb 



Another problem is that Literacies A and B should 
idofllXy b« defined in terns of specific items, not nec- 
essarily the sane items NAEP aggregated to create its 
reports. Onoe a group has created such a definition, it 
is unlikely that the pool of items will match the pools 
used to oeloulate the results presented in published 

2f^L'**P°'**^** Group A, for instance, may well find the 
Right to Read/HAEP definition of "functional" literaoy 
different from its definition, requiring new calcula- 
tions from new data. Steps are underway to facilitate 
this kind of analysis through improved data access sys- 
tems. Another approach might be to establish special 
literaoy paokages that, though administered along with a 
full essesement, are scored, analyzed and reported sepa- 
rately. 

Another diffioulty groups are likely to have with 
NAEP data Is that NAEP percentages are not referenced to 
oriteria or desirable levels of performance. A mean of 
55% may be oeuse for optimism or despair, depending upon 
how one feels about the importance of the exercises it 
inoludes. The exercises used for the functional I iter- 
floy study (MAFL) represent an exception to this rule 
beoause they were first judged to be items all 17-year- 
olds should know and then referenced to the real per- 
fornanoe of a group of excellent readers. In order to 
wrest polioy Implications from a single collection of 
HABP Itemsi one would have to establish criteria of some 
sort. 

NAEP .baokground variables, may not always be the 
ones literaoy analysts would use. Groups A and B would 
probably be interested in different sets of NAEP varia- 
bles and would probably want finer breakdowns of some 
results than NAEP can provide with its present sample. 
In additioni they might well want information about mi- 
nority^groups about whom NAEP has no data, agaih because 
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of aampX« size. 

Thtst eonalderatlona auggeat that within the pre- 
aant nodal, the National Aaaeasment could have even 
grtatar utility for Xlteraoy atudlea If aome of the fol- 
lowing matsurea ape taken: 

1* Maaaurea have already been taken to improve 
ganaraX aooeaa to raw NAEP data and materlaXa. 
In additlony aome effort to Index aXX NAEP ma- 
tariaXapotentlaXXyreXevant to Xlteraoy atud- 
iaa wouXd be moat; uaefuX. The buXk of the 
itasta wouXd be In the areaa of reading, writing 
and mathenatlea, but nany wouXd aXao be drawn 
froB- other aaaeaanent areaa. Such an Index 
oould conatltuta a ahopplng Xlat with whloh one 
eould conatruot and expXore a definition of 
Xlteraoy. Perhaps thla la alnpXy an expreaalon 
of an overaXX need for a NAEP exercise cXaaal- 
fioatlon ayateni that cuts acroaa aubject areaa 
to categorize exerolaea aXong nany different 
dinanslona. 

2. Poat hoc deflnltlona of Xlteraoy preaent probX- . 
ana that do not exlat If Xlteraoy (or llt- 
araolea) are defined prior to aaaeaanent. 
Given aufflclent reaources, the Aaaeaanent 
oouXd conduct a periodic Xlteracy. aaaeaanent 
consisting of reading, writing, oonputlng and 
other exercises, aXX of which wouXd be taken by 
the aame student or aduXt." the present atruc- 
tura of the Aaaeaanent provldea reading, writ- 
ing and mathenatlea data for different yeara. 
Thay are not aaaesaed together arid no alngXe 
student perforna taaka In aXX three areaa. A 
continuing aaaeaanent aXong the Xlnea of the 
AduXt LeveX Performance Project (1975), but 
avoiding some of Ita pltfaXXa (Fisher, 1978), 
oouXd prove uaefuX. 

3. ConauXtanta InvoXved in the creation of objec- 
tivea couXd be aaked to dealgnate certain aub- 
cbjactlvea aa reXevant to aone . notlon( s) of 
Xltaracy in their area. Each aubject area 
oouXd then have a apeclfic Xlteracy coiAponent 
that oouXd be reported upon aeparateXy. 



't. Aa it haa with materlaXa from the National Lon- 
gltudlnaX Study, NAEP can IncXude Itema, aur- 
vaya or varlabXea from other atudlea. Thla 
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would tlght«n the links b6tw««n NAEP and other 
sources of Information, broadening the context 
within whloh the data oen b« interpreted. 

5. As resouroea permit, haEP oan experiment with 
alternative measures of Xltereoy and help de* 
termlne to what degree partlouXar approaches 
are assessing literacy, thinking, world knowl- 
edge. Intelligence, reading or all of these. 
NAEP can contribute to studies cf the validity^ 
approprlatenesls, utility and reXlabUlty of a 
number cf Insti^uments, scoring guides and pro- 
cedures, along lines suggested In the next sec- 
tion. 



HAEP as a Proving Ground 
for Literacy Measuree 

A common complaint against literacy measures Is 
that few of them were designed to assess literacy per s e 
and many of them are flawed (Nefzlger, i976; Wisher, 
1978). If there Is to be sustained research Into liter- 
acy and continuing' measurement of progress, policy 
makers need more sophisticated Interrocking measures. 
The National Assessment Is a nature! proving ground for 
some of this new technology. Obviously it Is not pres- 
ently well suited for Instruments that require experi- 
mental control, extensive cne-to-cne Interviewing, In- 
tact classrooms, naturalistic nonschooX settings or 
lengthy tasksv Mor Is It a suitable vehicle for elabo- 
rate Investigation of individuals or Inquiries better 
conducted as small group studies. It is, however, a 
developer of large exercise pools used by a great many 
people. It has pioneered measurement approaches either 
too risky or expensive for commercial testers or re- 
searchers. When Integreted Into the larger network of 
literacy research, data gathering and testing, ifc oan 
try out new technology while Increasing Its o*m accuracy 
and utility. 

Table 1 lists 10 foci for measurement concerns In 
literacy and sketches NAEP's past and potential contri- 
butions. Although the ideas In the tsble cannot be dis- 
cussed In detail here, several deserve brief comment. 
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TABLE 1. NAEP Contributions to Literacy Measurement 



General co»Pr«Kefi5lon PfO* 
Ctssesi linguistic abilities 
■nd writing skills of the 
fluent reider 



^ Purposeful use of te^t In 
realistic suctions 



1. Differentlil perfomance 
on reading subSklUs 



HAEP EKi)er1enCe 
* Very Mttle 



Prob 



ScMDi* SklH/scan 
study* functional 
literacy study^ 
study skills qties- 
tlonsi Etepartoent 
of libor Itaat 

Extensive. First 
two reeding 
assessments 



• Separation of read1n9 and writing 
aisessnents 

" Reteerch goil difficult to Inte- 
grate Into broader asseisaent 



Creiting realistic sltti^tloos 
Controlling for niMroui viri* 
ablei 



Theoretical problems wIM lub* 
skills and their relationships 
Hatrix saiple helpers aitelyses 
Eteflnltlon of subikllls contrcH 
verslil. 10 skills defined for 
one etsestMnt ire rejected at 
next assessatnt 



Possible fetlons 

• Assess love raiding and witing 

" get her 

• Af^lyze literature essey perforin 
ai^e by raadlng ^scorv** In saM 
package 

• Define fluent readers ind cross- 
tabulate on background factors 

• Design speclil package for study of 
flti^t reeder 

• Iiposslbte to do within usual 
assessment constraints* Requires 
tightly controlled experlnent or - 
observitlonal study 



• Meinlngfulness end i«p11Cat1ont of 
remits In itoubt. NAEP<s Read/lit 
Advisory Cowlttee could sponsor 
dibate. request |»pers on this sub- 
ject 



Affective components of 
literacy 



Soaei A few ques^ 
tlons about ittl^ 
ttHfeS toward 
reading^ litera- 
ture, writing 



Self report daU 
Test situation 

Hatrix sample and saqile depUi 
Halt analysts 



Experiment with varieties of ef- 
fective aeisures (e.g. I semantic 
differentials, branching Sor^ 
veys* one-on*one Interviews) 
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TABIE 1 " Continued 



7, 



Heasureaent Focus 

Vrittng skitU beyofK^ 
those assessable by 
nuttlple-dtoice tests 



HAEP Experience 
- Extensive 



Prebti 



Influence of motivation 
on reading and writing 
skills 



Literacy in work/life 
settings 



None 



Costs I logistics of open-ended 
scorlngi complex analyses pre- 
sently kHp NA£P from fully 
analyzing Its writing data 



« Survey situation with no op-, 
portwiity for feedbaci: or pro- 
longed observation. Hust rely 
on respondents' essessnent of 
their «ot1vet1on 

- No teacher Judgntnt about pupil 
mtlvatlon evailable 

• Test situation rules out obser- 
vational 4tata 



Possible Actions 

- Experipentatlon with various holis 
tiCk rtibrici prinary-trait measure 
end techniques for analysing co- 
herence, syntactic fluency, etc- 

• Hore valiwitlon of scoring pro- 
cedures presently used 

- NAEP/NIE sponsored syiywsla on 
neasurenent of writing 

- Increase hud^t for writing suf- 
ficiently to allow full-sajifkle 
scoring 

- Research studies on scoring of 
essays over time 

• Kalie NAEP's essay bani: fully avail 
able to secondary researchers 

- Experiioental background questions 
ained at assessing motivation 

• Interviews using branching ques- 
tions {as in Vears 01 and 02) 



Experliient with background question 
Coinplement regular assessnent with 
several coontinated case studies 
Conduct a special work/literacy 
assessnent 
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TABLE 1 — Continued 



Heaiuratent Focus 



HAEP Ewpartence 



Prob]< 



Posilble Actions 



8« Specific effdcti of rMd- 
\ng and irrtting and 
Mtency theory up<w 
assesiaent Initrunents 



• Cittenslve, but 
undeveloped 



9. New AMiures » of dtten- 
tlon; ptrcepttofi} nenory; 
content OMpectatlons; 
AMarenait of 1ncon9ru1t1e5; 
■bllUy to predict* ctMck 
and revise text aeanlngs » 
that art useful tn a sur- 
vey context 

0* Association of background 
diaracterlstics iiKl 
ich4eve«ent tn various 
literacies 



- Very little 



CMttnslve work Ub* 
ulating and cross- 
tabulating back- 
ground factors and 
achleveeient. Less 
OMperlenco p«r- 
fomlnfi mtytica] 
studies 



NAEP has developed reading assess- 
ments under different theoretical 
tiltrellas. But did psycho1lngut5t1c 
theory lead to e different kind of 
readlfig Instrueent frooitdilch dif- * 
ferent kinds of Inferences cen be 
drann? If so» In what specific ways? 
If pott vby hot? lAet Inferences 
can properly be drawn ebout writing • 
froA prlHry-tralt Measures? Whet 
are primary traits* textually? 
Bnat an the rules for generating 
prlwry^trBlt definitions and dis- 
tinguishing ''real" ones fron 
**unreal*? 

These factors In reading perfon* * 
ance have received attention In 
research but have not been trans* 
lated Into test or assessMent 
1tflKs> So«e of the problevis are 
obvious* but mst of thOM are 
unknown 



Seople depth and design rule out 
certain kinds of analysis 
Background factors conpete with 
ach1eve»int neasures 
Logistics/cost 
TlaellnesB 



Studies of the HAEP OMnrclse 
developiDent process and Its effect 
upon the translation of theory 1nt< 
aeasurenent instnnents 
Secondary research Into psychoMtrl 
differences between assessitfnt 
tools developed frtM differing as- 
swptlons 

Syvposla on underlying principles 
of prinary-tralt definition* legitl 
■Izatlon and jneasurenant 



Closer links between asiessMnt de- 
velopers and researchers 
Synpoila devoted to Identifying 
these research findings with 
gr^test proeilse for assessntent 
tectoiology 

Systeaitfc eMperlnentatlon with 
new Beasures 

ftevanped end enlarged sample 
Expanded secondary research pro* 
gran 

Edifcatlon progran for users 
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The most obvious probXeo Table 1 Illustrates Is 
that many of the queatlons people have about llteraoy 
are research questions first, measurement questions seo- 
ond and appropriate large- so ale assesatsen t queatlons 
third. The route from the question **What mental pro- 
cesses underglrd fluent reading and writing?** to Items 
designed to answer the question **How many nine-year-old 
fluent readers and writers show evidence of using mental 
processes A and B?^ Is a oomplloated route. Small-scale 
research must unearth some relstlvely strong hypotheses* 
Someone must design reliable and valid measures relating 
directly or In a ohaln-llke way to the hypotheses* And 
someone else must find ways to fit questions originating 
In experimental research Into a descriptive survey with 
a very different method elegy t no experimental controls 
and numerous goals besides research* This Is what the 
comment ^Research goal difficult to Integrate into 
broader assessment** means In the third column after 
focus #1 . 

In spite of difficulties^ however, there Is no rea- 
son to believe that a number of research questions can- 
not be converted to assessment questions or scoring 
schema as part of a long-term effort to Improve the pre- 
cision and accuracy of literacy measures. NAEP*s writ- 
ing assessment! for example, Illustrates a very produc- 
tive marriage of research Interests and assessment tech- 
nology. There Is currently considerable theoretical and 
research Interest In language cohesion and syntactic 
fluency* NAEP Is drawing heavily upon that activity, 
applying to national samples schema developed on very 
small samples or never previously applied to any essay 
samples at all* Information from the third writing as- 
sessment will both clarify changes In writing and clar- 
ify the utility of new approaches to the scoring of 
open-ended questions. 

Some comments about each measurement focus In Table 
1 and the problems It raises; 

Poous ^* NAEP*s separation of reading and writing 
poses difficulties for finding out more 
about the characteristics of people flu- 
ent (or not) In both areas. 

Pocus 2. Certain literacy observations should be 
made In a realistic context rather than a 
test-like atmosphere* NA£P has assessed 
skimming and scanning skills and has 
asked students to Imagine specific writ- 
ing situations, but there are obvious 
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Xlffllt9 to the klnda of raaaaurss It oan 
anploy In this area as It la praaently 
daaignad . 



Fooua 3> ProfesalonaX dlaagreamant about lltaraoy 
aub^klXXs or ooniponanta affaota raaaaroh 
and aaaaaaniant both. To a X 1ml tad da- 
gr00^ KAEP ahouXd axperlnant with aome 
aubaklXX niaaauroa In aaoh assasamant and 
anoouraga mathodoXogloaX and thaoratloal 
raaaaroh using NAEP data tapas. Of aonia 
Intaraat hara Is tha fact that aX though 
thara la dabata about phonloa In tna 
raadlng profaaalon and dabata about gram- 
mar In tha writing flaXd, KAEP aaaaaaaa 
nelthar phonloa nor knowXadga of grammar. 
Inataad, KAEP aaaaaamanta aim to maasura 
tha ovaraXX aklXXa (oomprahanslon and 
writing) of which thasa ara oomponanta. 

Fooua 4. Studlaa of Xltaraoy that proba for motl- 
vatlonaX faotora ara aomawhat poaalbXa 
within a Xarga-aoaXa asaaaamant frama-> 
workt but thay ara Xaaa than IdaaX. Stu- 
danta oan ba aakad dlraot quastlona about 
how motlvatad they ara to do oar tain 
things. Tha Information oouXd not ba var- 
If lad by observation or by taaohar re- 
ports alnoa KAEP doea not aaaeaa Intaot 
oXaaarooma. In tha earXy year a t KAEP 
uaed a ona-on-one Interview to gather 
aupportlve data. However ^ the technique 
waa abandoned aa too ooatXy before It waa 
perfeoted . 

Foous 5. Within Ita present oonatralnts KAEP oouXd 
aak at ud ants to data IX their Xlteraoy 
neada In and out of aohooXf but thla ap- 
proach haa obvious drawbaoka. Case stud- 
lea » Integrated Into a fuXX asaeaament , 
have been suggested aa a poaalbXa means 
of enriching KAEP data. 

Fooua 6. KAEP routlneXy aaka questions about atti- 
tudes. Thla la an area where conaldera- 
bXe experimentation and Improvement 
shouXd be poaalbXa. 

Focus 7. KAEP has experimented extenslveXy with 
ways to mlXk data out of essays. The 
primary constraint In this area has been 
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Insufficient money to analyze full sam-* 
pies. Consequently T NAEP has less data 
about the writing of blacks t Hlspanos and 
other groups than it should have. 

Focus 8. Theoretloal problems In the measurement 
of literacy are as Important as the prao-^ 
tloal problems. NACP needs theoretical 
studies of such or It leal Issues as the 
oonatruct validity of assessments devel- 
oped under psychollngulstlo theory or the 
underlying nature and Independehoe of 
primary traits. NAEP^s Ideas need more 
exposure and scholarly debate. 

Focus 9- Symposia designed to tighten the linkage 
between researchers In llteraoy fields 
and assessment developers would benefit 
both groups. 

Focus 10. HAEP has extensive exper lenoe oross- 
tabulating background oharaoterlstlos and 
achievement In various llteraoles. The 
analytlo possibilities ( as opposed to 
cross- tabulation) of NAEP data will be 
ex pi or ad by staff and second ary re* 
searohers through the new KIE secondary 
research program. Although there are 
analytlo opportunl yet to be taken 

advantage of, there e also limitations 
to what oan be done with the current NAGP 
sample and the current approach to pack- 
aging and admlnldtarlng Items. 

Table 1 raises only a few of the oonslderatlons 
Involved In using NAEP as a proving ground for measure* 
ment advanoes. The reader Is Invited to ponder and ex- 
pand upon It. Table 2 displays some of the approaches 
to llteraoy measurement being employed In the current 
reading/literature assessment* Note that although there 
are multiple-choice questions for eaoh of the areas of 
aonoernr there are also open-ended f short-answer and 
essay questions, as well as survey questions aimed at 
gathering data about suoh things as reader self-concept 
and attitudes. 



27 

32 



TABLE 2. Year 11 Rsadlng/Llterature Assessment Literacy Measures 



Assessment Area 



unctlonal tasks/skHls (e.g., schedules, maps, 
phonebook, want ads, recipes, bank check 
application forms) 

;tudy skills (e.g., skim/scan, maps, charts, 
graphs. Indexes, tables of content, dic- 
tionary pages, card catalogs, encyclopedias) 

leading comprehension (using lexlcal/proposl- 
tlonal/textual approach, literal/Inferential 
approachi and explicit/Implicit continuum 
approach) 

Higher-level" cognitive/affective responses to 
written works (evaluate, analyze, generalize, 
draw Inferences, emotional responses) 

eading comprehension by genre/passage type (e.g., 
poetry, science, social science) 

eading comprehension as related to student 
background information such as experiential, 
attltudlnal, reader self-concept, achleveftent 
level and other demographic variables 

tudy skills performance as related to study skills 
background information 

1C means mtltiple^ohoiaet OE rrteane open-ended. 



^■ifgTA^^ Fo rmat/Heasurement Technique 
OE.* sbrt ' 



flCnisTna 
Accffliqsar^ing 
Handouts or 
Visuals 

X 
X 
X 



X 
X 



Answer or 
**F111-In" 



X 
X 
X 



X 
X 



UE* Self- 
Essay Repor 
(Force 
Choice 
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Lltftraoy Issues HAEP Could Address 
Only With Hajor Design Changes 

Table 1 brings to light a number of research ques- 
tions about literacy that oannot be answered well or 
efflolently within the current NAEP framework. This 
raises the question of whether the Assessment should be 
resheped to be a better research tool or whether re* 
searchers should lower their expectations about what can 
be done within a given assessment design. In a 1970 
discussion of the disappointing results of large-scale 
program evaluetlons, David Cohen clearly recognizes the 
conflict between a census- like approach to evaluation 
and a research approach! 

By definition a census measures stasis i It 
quantifies how things stand. If done well^ It 
can reveel a good deal about the inter- 
connection of social structure; if It recurs , 
It can throw much light on how things change. 
But no census can reveal much about change 
other than Its patterns » probing Its causes 
and dynamics requires rather a different re- 
search orientation. And no census can produce 
qualitative data , especially on such compli- 
cated organizations as schools . . « . Using a 
census as the central evaluation device for 
large-scale multipurpose programs assumes that 
systematic experimentation is very nearly im- 
possible within the large operating programs 
and can best be carried on by clearly distin- 
guishing census from experimental functions « 
It would be foolish to Ignore experimentation 
— it should be Inoreeaed — but it would be 
Illusory to try to carry it out within pro- 
grams that have other purposes <Cohen, 1970t 
P« 120). 

The same distinction is made by Hosteller and 
Moynlhan < 1972) , Dyer < 1972) and others: Informational 
needs requiring different approaches oannot be met 
equally well within a single program. The policy ques- 
tion this fact raises is sketched in the Introduction 
but beers repeating here; Should the Assessment » de- 
signed plmarlly to provide long-temti census-like data 
about deep structure trends » be redesigned to permit 
more enalytlcal studies, experimental reseeroh and tar- 
getted program evaluation? What' are the trade offs in- 
volved? 
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There are oertalnly many questions about literacy 
that NAEP does not ourrently address. For Instance, 
researohers have very little data about the preolse 
level of llteraoy that Is truly required for personal, 
olvlo and eoonomlo "survival" In this oulture (Stloht, 
1971; Hunter and Harmon, 1979), We ara no longer even 
oertaln about the trulsn that llteraoy Is ralsted to 
thinking ability — Sorlbner and Cole (1978) have oast 
doubt upon several versions of that belief. Indeed, the 
preolse funotlons of llteraoy akllla In different kinds 
and levels of soolety are poorly understood. At the 
nonentf anthropologloal , ethnographlo studies appear to 
hold most promise for olarlfylng these and similar nat- 
ters. A o*nsus-llk« survey like NAEP oould systenatl- 
oally gather attltudlnal or desorlptlve data bearing on 
these Issues by asking people how muoh reading and writ- 
ing they do, how Important th«se skills are, and so on; 
oonoelvably, the Assessment oould even oross-valldate 
ethnographlo oase studies by looking for similar pat- 
terns of llteraoy aohlevement In similar demographlo 
units and populations. But these efforts would be sup- 
portive only and would not be oompatlble with the level 
of detail oharaoterlstlo of ethnographlo rasearoh. 

The ourrent Assessment Is In an equally weak posi- 
tion to deal dlreotly with fundamental questions about 
how llteraoy Is aoqulred or how It should be taught. 
Aesearoh Into the ecqulsltlon of reading and writing 
skills has served to generate as many questions as an- 
swers. Models of the reading end writing prooesses be- 
oome Inoreaslngly elaborate and oomplex with eaoh study 
of the psyohologlcal, oognltlve, llngulstlo, semantlo or 
aoolal aspeots of llteraoy aotlvltles. Clearly, the 
family, the soolety end the sohools play important roles 
In the aoqulsltlon of llteraoy skills. But exactly what 
those roles are and how they interaot for different In- 
dividuals or groups we do not yet know. Aesearoh In 
suoh diverse areaa as ohlld development, oognltlve psy- 
ohology, egopsyohology, rhetorloal analysis, llnguls- 
tlos, semlotlos and tha soolology of eduoatlon Is moat 
likely (If It oen be brought together ooherently) to 
olarlfy our underatandlng here. The National Assessment 
as presently oonoelved would play, at best, minor, Indl- 
reot roles. For Instanoe, it might spot areas or groups 
that display rapid ohanges In llteraoy skills. Follow- 
up studies of such situations oould ooncelvably unearth 
Important variables or oontexts that elude deteotlon by 
other means. 
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But clear information about how literacy is ao- 
quired or best taught does not QOme from oroas-tabuXated 
data. It comes from careful experimentation with pains- 
takingly ohosen and characterized samples of people in 
settings permitting the introduction of specific varia- 
bles p control groups, follow-up studied, and so forth. 
This would require a very different NAEP. 
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PAHT 2 
EQUITY 



Introduction 

History, a rich tradition of political philosophy 
and casual observation converge powerfully to suggest 
that arbitrary social Inequities* arise as people create 
political and aconomlc systems with which to balance 
diverse needs for power, status, food, happiness and the 
Ilka; and that societies tend to perpetuate these une- 
qual relationships by creating them anew from one gener- 
ation to the next. Central to the process of perpetuat- 
ing or modifying social Inequities Is a soclety*s educa- 
tional system. Thus, it Is no surprise that battles 
over Inequities in America are often fought on educa- 
tional tur f « But It should also be no surprise that 
such battles have seldom end ed with unambiguous victo- 
ries or defeats. Charged both with educating and sort- 
ing children , the American educational system contra-^ 
diets Itselft constantly betrays one or another of Its 
constituents and can move only haltingly toward goals 
society endorses one year and retreats from the next. 



*It Is useful to distinguish the word Inequality 
which suggests unequalness from Inequity which 
suggests unfairness . The argument that all men are 
philosophically equal Is often misunderstood to Imply 
that all men should be physically, morally or Intellec- 
tually Identical; social programs attempting to Insure 
that people will be treated equally are then seen as 
attempts to Insure that people will be made equal with 
respect to all human attributes. 

The word Inequity removes that confusion by focusing 
upon a relation between Individuals and a system of jus- 
tice* To say that Inequities exist Is to say that cer- 
tain groups or Individuals are not receiving equal 
treatment under laws Intended to apply equally to all 
citizens* The existence of Inequalities Is not neces- 
sarily a bad situation; the existence of Inequities, 
however, almost always Implies a miscarriage of justice. 
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Equity studies In American education should past on 
a petXistio model of this complicated system. Suoh a 
model must account fop a multitude of deoXaptd and un- 
deoXaped ppogpessive and pegpessive ppessupes that de- 
fine education's pupposes unevenly at diffepent times. 
It must account fop education's poXa in sooiaXizing, 
sopting and stpatifying young peopXe; incuXoating atti- 
tudes compatible with a competitive eoonomio system and 
a histopy of some inequities; tpaining them in vepy 
basio skiXXs; impXementing sociaX "pafopma" that othep 
institutions do not wish to suppopt; and — most impop- 
tantXy — educating chiXdpen in ways that enhance theip 
seXf-esteem, stpengthen theip abiXities to think 
cXeapXy, opiticalXy and cpeativeXy, and enooupaga tham 
to tpanaoend in theip own ways any fopoea that might 
disooupaga theip gpowth op unfaipXy Ximit theip fpaedom. 
Within that Xast function, a pealiatio modeX muat aXso 
account fop the uniqueness of individuaX Xaapning and 
teaching styXes, the divepsity of enviponments end op- 
poptunitias for Xeapning and teaching, the pichneas of 
oXaaspoom intepactiona and the diffiouXtiea of evaXuat. 
ing aXX this with instpuments that are both peduotive 
and vuXnepabXe to the same bXindnesses that pepmeate the 
system being studied. 

Heseapoh based upon an inadequate theopetioaX modeX 
is XikaXy to discovep inequities that ape tpiviaX op 
ovapXook acne that ape opiticaX; it ia aXso XikaXy to 
Xead to simpXistic concXuaions about the affaotiveness 
OP ineffectiveness of schooXs in genapaX, eapeoiaXXy if 
its methods ape too peductive op simpXiatio, Thus, it 
ia incumbent upon poXicy makeps in education that they 
have a synoptic view within which the pXace of each pe- 
seapoh study op evaXuation. is as cXeapXy fixed as possi- 
bXe in peXation to evepy othep one. This is a taXX op- 
dep . 

That no such view cuppentXy exists is obvious to 
anyone who ventupes into the Xitepatupe on equity. As 
with Xitepacy, thepe ape confXioting concepta invoXved , 
con fXic ting data bases and confXicting findings. It 
might be usefuX to peview an assoptment of equity 
tabXea, gpaphs and statements in opdep to undepstand 
concpeteXy the miXieu in which NACP data exist and the 
probXams NACP faces in defining its most usefuX roXa in 
that raiXieu, Appendix B both estabXishes that miXieu 
and documents the pervasiveness of soma inequities in 
American society. 
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Trends In Equity and Equity Studies 



The figures in Appendix B reveal rauch about both 
equity and equity studies. About equity, one learns, 
among other things, that: 

• Considerable Inequities in eduoatlonaX achieve- 
ment, eduoatlonaX attainment, oocupatlonaX at- 
tainment and Income exist across Amerloan soci- 
ety, and Important groups such as women, blacks, 
Hlspanos, Amerloan Indians and the poor receive 
disproportionately low shares of educational and 
material wealth. 

• Blacks and whites have different perceptions of 
educational opportunity, as do Northern and 
Southern blacks. 

• Vfhltes* attitudes toward blacks have Improved in 
the last 20 vcara (NAEP citizenship data tend to 
support this) . 

• Lower percentages of blacks and Hlspanos than 
whites finish high school and college, but the 
differences among the races in eduoatlonaX at- 
tainment are shrinking. 

• Increased equalization of educational attainment 
does not appear to have led to Increased equali- 
zation In occupational status or income. Al- 
though subgroups of women and minorities have 
made gains in professional fields and Income « 
the overall positions of women and minorities 
relative to nen and to whites have remained vir- 
tually the same. 

• Educational attainment and Income for Indi- 
viduals seem to relate more to social class and 
background and unknown factors than to such 
things as academic achievement or general intel- 
ligence . 

• IndlansS blacks* and Hlspanos* achievements are 
below whites* achievement 1 evels , on the aver- 
age. 

• Achievement differences between minorities and 
whites are considerably less for certain sub- 
groups such as those minorities coming from 
homes In which at least one parent graduated 
from high school. 
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Various of these statements oan be oomblned to nake 
suggestive speoulatlons. For Instanoe, It appears as If 
whites* attitudes toward minorities night oontlnue to 
Improve over the long run. It appears as If Improve- 
ments In minorities' eduoatlonal attainment will eventu- 
ally be refleoted In Improved aohlevement levels for 
their children. It appears as If Improved educational 
aohlevement and attainment will not have much direct 
effect upon the relative occupational status and Incomes 
of women and minorities. It appears as If schools, as 
they have been supported and run so far, will not do 
much to change nonsohool Inequities. But the data are 
only data, and If conditions change, one'a Interpreta- 
tion of them might change as well. 

This sprinkling of examples In Appendix B Is also 
meant to suggest how diverse are the definitions of eq- 
uity, the sources of data used to assess It and the 
methods of aggregating, analyzing and Interpreting It. 
Equity hes been defined In terms of access to school 
resources, opportunity to take advantage of resources, 
aohlevement In the basics, full educational achievement, 
eduoatlonal attainment, aooess to the professions and a 
more equitable distribution of wealth. Equality of edu- 
cational opportunity has been thought of as providing 
adequate school resources to enable children from dif- 
ferent backgrounds to reach the same levels of academic 
achievement! providing Individualized Instruction enabl- 
ing each child to fulfill his or her potential; removing 
racist and sexist attitudes and practices from the 
schools; providing compensatory programs for minority 
children; and all of the above. Data sources Include 
simple attltudlnal polls, school surveys, census and 
labor statistics, standardized tests and personal testi- 
mony. Methods of analysis run the gamut from fiction to 
simple descriptive surveys to multlvarlete path analy- 
ses. Each data source has Its own measurement limita- 
tions and Its own anchor In a particular sample and 
point In history. Each analytical approach has Its own 
characteristic blind , spots. What we understand about 
equity we understand by somehow connecting these dispa- 
rate inquiries to arrive at a general Impression. 

The entire matter is further complicated by the 
fact that equality of eduoatlonal opportunity Is both an 
eduoatlonal and a scolal goal. That Is, equal eduoa- 
tlonal programs aim not only at changing education but 
at changing society through education. Two consequences 
flow from this fact^ THe first Is that equity studies 
are as political as they are solentlflo. Educational 
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infppmatlon prejjented within a poXltloaX context that 
threatens existing powap and status relationships wHX 
be piepoelved and responded to poXltloaXXy. riot rational- 
ly. The second oonsaquerioe is that aduoacionaX Informa- 
tion aXonewlXX never be sufficient to judge the suocess 
of a soplaX goaX. It is quite'posislble tor Inequities 
to diminish while measurabXe aduoatlbnaX outcomes for 
minorities remain at a constant XaveX. It is equaXXy 
posslbXe for Important Inequities to persist even though 
neasurabXe educatlonaX outcomes for minorities Improve. 

UnX Ike pure research driven by the curiosity of the 
researcher, equity research has been driven prlmarlXy by 
InstitutlonaX poXlcy needs and major soclaX poXlcy ques- 
tions. When tha courts defined equity in terms of equaX 
edupatlpnaX opportunity, Interest focused upon finding 
those school factors most responslbXa for dlfferentlaX 
academic performanoe so that their avalXablXlty couXd be 
equaXlzed. The question at hand dictated the methodoX- 
ogy — various kinds of factor ariaXysls of schooX and 
nonschooX inputs thought to beer on performarice* Sev- 
eraX major researohers (coXeman , l9o6; Jencks, 1972 1 
BowXes and Glntls, 1976) found the nonsohooX varlabXes 
they seXected contributed more to the aohlevement meas- 
ure they seXected th^n did the schooX resources they 
selected, spawning reaneXysas of the data, new studies 
and a widespread misapprehension that ''sohooXs don't 
make a difference.** 

The trend In recent years has been either to try to 
show these researchers **wpong** by discovering signifi- 
cant schooX factors or to further cXarlfy the nonschooX, 
structuraX factors that contribute to InequaXlty, In 
either case, researchers empXoy muXtlvarlata anaXyses 
liberally In order to create causal modeXs or reveaX 
unsuspected reXatlonshlps with potentlsX poXlcy reXe- 
vance. 

At the same time, however , equity studies have 
drawn heavily upon desorlptlve data bases such as the 
U.S. census and the U.S. Department of Labor statis- 
tics. In On Equality of EducatlonaX opportunity . 
Hosteller and Noyntnan n97z) lOOK bo naEP as a correc- 
tlve to some of the flaws Inherent m the "one shot" 
Coleman study. Not only could NAEP provide the trend 
data Coleman sorely needed, but It could be a clean 
source of group achievement data and it could offer a 
much broader range of meaaures than the limited stan- 
dardized tests Coleman used. Henry Dyer, elsewhere In 
the same book, terms NAEP a useful ''descrlpislve'* study, 
which he distinguishes from analytical and experimental 
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studies alm«<) at establishing Input/output relationships 
•nd evaluating interventions. 

Thus, NAEP can contribute to our understanding of 
•quity problems both as a source of descriptive data and 
as an opportunity for certain kinds of analysis. 



HAEF'8 Contributions to Equity Studies 
as a Source of Descriptive Data 

Certain groups may be receiving more hours of In- 
struction than others, better books, preferential treat- 
ment, more encouragement to pursue certain career op- 
tions or to aim for college, inequities might show up 
In cognitive achievement but not In attitudes. In mathe- 
matics but not In music. In writing mechanics but not In 
prose expressiveness. Hlspanos In certain communities, 
types cf schools or socioeconomic situations inay be per- 
forming certain kinds of tasks better than all Hlspanos 
cr ether groups. We. cannot predict where Inequities 
might crop up or which ones wHX prove to be of critical 
Importance to the next generation, it may appear today 
that inequities In reading comprehensloh -achievement 
levels are of greatest importance, but 10 years from now 
the public, may be- more concerned about Inequities In 
positive social attitudes. As long as the schools per- 
form mahy different functions and teach a variety of 
subjects crucial both to later life employment and later 
life enjoyment, an ideal assessment should remain broad. 
Will the elimination of significant group differences In 
loy-level reading skills be a worthwhile accomplishment 
If the price Is greater group differences In understand- 
ing history, the arts or civic duty? A broad descr*lp- 
tLve base offers the best hope of collecting the data 
needed in the future In order to address such issues. 

The current Assessment describes the performance of 
Important groups such as blacks, Hlspanos and women. 
However, the sample Is not large enough to Include Ori- 
entals, "boat people" or any number of group? that cur- 
rently meet with unequal treatment In the sch' 9l3 or may 
one day be policy relevant. Nor Is NAEP' s - iple large 
enough to permit more precise characterlzax . ns of the 
groups It now reports on. When Interest s.ifts from 
"blacks" to "blacks la the Southeast," NAEP can be help- 
ful. But a further shift to "blacks In the Southeast 
whose parents attended college," or "suburban blacks In 
the West," strains NAEP's present capabilities. Equity 
studies are moving increasingly In this direction as the 
economic and human Importance of making such dlstlnc- 
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In addition to broad coverage of subjects and 
groups, the Assessment must also aim for coverage of 
desoriptlve variables that both order the data In slg;- 
nlfloaint waya and facHltiate analytical studies. Table 
3 llata the background variables oolleoted In the Year 
11 readlng/llteratojre assessment. Most of the variables 
break Into many categories. For Instance, there are 
nine oanaua regions, eight oategorles of television ex- 
posure and seven methods of Identifying students for 
remedial reading programs. When all these disorete 
oategorles are added together, there are more^ than 100 
sohool-level and more than 50 student- level variables 
for 17-year-olds, thus, the results of any given read- 
ing exerolsa oould conceivably be reported In terms of 
more than i50 groups of schools or people. In addition, 
any given exercise, exercise part or cluster of exer- 
olses oould serve as a variable by whloh to examine re- 
sults of any other exercise, exercise part or oluster. 
Slnoe there are more than 450 Items In the 17-year-old 
reading/ literature assessment, there is a conceivable 
600 X 600 variable matrix before one even begins to cre- 
ate variables by clustering, if NAEP did nothing more 
than tabulate these results over time it would still be 
performing a unique and invaluable descriptive service. 

National Assessment's value as a rich descriptive 
data base Is determined by the Incluslveness of Its as- 
sessments and variables, the descrlptlveness of Its var- 
iables, the amount of access people have to the Informa* 
tlon and the amount of cross^tabulatlon Its sample per- 
mits. Eaoh faotor deserves brief comment. 

Inoluslveness . The more equity relevant groups and 
variables Inoluded in the Assessment battery, the more 
relevant NAEP will be to equity studies. However, some 
Important trade offs must be kept in mind. An overly 
elaborate principal's questionnaire could affeot either 
sohool par tloipa tlon or the accuracy of the data. A 
lengthened student background questionnaire cuts Into 
assessment time because every minute spent answering 
baokground questions Is a minute not spent answering 
aaaessment questions. Oversampllng minority groups for 
Inoreased praolslon can lead to deollnes In the preci- 
sion of national estimates. And , of course, Incluslve- 
ness la not a virtue In Itself; the trick Is to Include 
the right variables. Presumably, smaller-scale studies 
would be the source of new, potentially fruitful varia- 
bles and prlnolples of organization or tabulation. 
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TABLE 3.^ Background Variables Collected In the 
Year 11 Reading/Literature Assessment 

Data Source 
School Student ECS 



Official Staff 
All ages 

Region (4) X 

Census region (9). X 

Public/private control of school X 

Enrollment and average dally 

attendance x 

Comnunlty size x X 

ComnunUy type x X 



School occupational profile X 

School racial/ethnic mix X 

Title I eligibility; number of 
students served x 

Library size, accessibility X 

Sources of reading material X 

Instructional methods and materials X 

Availability of In-service train- 
ing X 

Remedial reading program X 

- Percent of students served 

- Type of personnel 

- Methods of student Identification 

Reading enrichment classes X 



Educational materials at home X 

Parents' education level x 

Television exposure x 

Bilingual home X 
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TABLE 3 ~ Contimad 



Race 
Sex 

Grade In school 

Birthdate (monthtyear) 

Kind of reader 

Easy/hard reading tasks 

Kindergarten attendance 
Ages 13 and 17 

Time spent on hotnework 

State of residence at age 9 
Age 17 only 

Required English instruction 

Curriculum offerings (language arts) 

Advanced placement* honors programs 

SES Index (things in the home) 

Mobility Index 

Family size, birth order 

Race 

Instructional methods and materials 
State of residence at age 13 



Data Source 
School student 
Official 



X 
X 
X 

X 
X 
X 

X 
X 

X 
X 
X 

X 
X 
X 
X 
X 
X 



ECS 
Staff 



HO 



i5 



Dftsorlptlveneaa > To the extent that any background 
questions are anbiguouei out of date^ too abatraot or In 
any other way remote fron reaXltyt the data they gener- 
ate will be that much less useful. In addltloni survey 
questions have well-knovm limitations that effeot the 
renge of Inferenoea that oan safely be drAwn from them. 

Descriptions of the world are Judged aooordlng to 
their oompletenees and their sensitivity to Importent 
distinctions* There are many different kinds of blacks. 
Hlspanos and Indians In many different socloeduoatlonal 
settings. The more distinctions HkEf oan make about 
suoh groups^ the more useful It will be* in the pasti 
Hktf has oross*tebulated race by region^ race by Title I 
eligibility of school^ race by grade In school ^ race by 
percentage of white students In school i race by sex i 
reoe by parental education and e number of other varla<^ 
bias. Currentlyi NAEP Is exploring cross*tabulatlon by 
high* and low<^performanoe groups* The more cross clas*^ 
slflcatlon done I the more descriptive the data. The 
limit to eross olasslfloetlon Is^ of course^ sample slxe 
and stratification. 

Access to the. Data * It is one thing, to have mll- 
llons of data points to organise and cross*tabulate| but 
It Is another thing to absorb or work with them* Here a 
HAEP* strength a wealth of descriptive data Is also 
a weakness* There Is too much to deal wlth^ even after 
categories are collapsed and data are reduced to manage- 
able/ reportable proportions. Current efforts both to 
make raw data more accessible and to train others how to 
use It will helpi but as It agee^ NAEP will always need 
new technologies and programs for making the data maxl* 
mally available in mapy forms. 

Interpretation of Descriptive Data * All . the NAEP 
data gatnered so far could conoelvabiy be tabulated and 
oross-tabulated aooordlng to all Imaginable useful clas- 
sifications and made available to every Intereetad per* 
son In America. But the question of what such descrlp* 
tive date mean oan only be answered conditionally* NAEP 
data describe percentages of people who oan do certain 
things under certain olrcumstanoesi and changes In those 
percentages* They answer, a limited number of simple 
questions well^ e*g., **Is reading achievement declining 
or Improving?** But the data need to be placed In other 
oontextSi linked to other kinds of data and Interpreted 
In terms of other frames of reference to play their 
strongest role In shedding light on American education 
and culture* Interpreters of NAEP data must constantly 
be made aware of the difference between oross-tabulated 
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and expeplmentaX data. It is one thing to note that a 
sohooX with program A produoad higher performanoe than a 
aohooX with program B; it la another thing to experimen- 
tally introduoa program A Into equivalent sohooXs, o6n- 
troX aXX raXavant faotors and oonoXude that the program 
la rasponalbXa for higher performanoe. 

This Intranaltlvity of NAEP desorlptlve data re- 
quires partlouXar attitudes and approaohea to data that 
are oXoaar to those of tha historian than the experlmen* 
tax solantlst. Tha rasponslblXlty for cXarlflcatlon 
faXXs as haavlXy on ths inquirer as It does on the data 
aouroa. Humlnatlon, dlsolpXlnad observation and expXor- 
atory anaXysls osn pey dlvldsnds with descriptive data, 
though thay msy saem "unsolantlfle" to researchers who 
ere aooustorasd to axperlmantaX studies, hypothesis test- 
ing and oonflrmetory aneXysaa ( aee Cronbaeh, 1973; 
HosteXXar and Tukay, 1977; Tukey, i960, 1977; Burton, 
1 978 ) . 



NAEP*8 Contributions to Equity Studies 
as a 3ouroa of AnsXytlosX Date 

AnaXytloaX studies InvoXve searohlng for reXatlon- 
shlps between aspaota of tha adueatlonaX system and Its 
produots.' They are more deolslon-orlented (cronbaeh, 
1973; Dyer, 1972), more often tied to speolflc poXloy 
questions and more often dependent upon nuXtlvar Ite 
anaXvsea than desorlptlve atudles. NAEP's cross- 
tabulations have an anaXytloaX quaXlty, but It Is best 
to reserve the term for studies that aim prlmarlXy to 
aatabXlah dagraea of raXatlonshlp between oomponents, or 
prooedures (Xlke baXanolng), designed to correct for 
dlsproportlonaXltlas of group oharaoterlstlcs among the 
groups NAEP aampXea. 

Baoauae fuXflXXlng NAEP'a descriptive mission eco- 
nofflloeXXy oaXXa for a atratlflad muXtlstage cXuster sam- 
ple with unequal probablXltles of seXectlon, its compXex 
(olustarad, weighted) data base does not meet the random 
sampling assumptions underlying many statistical analy- 
sed. Aa a oonaaquanoe, traditional analytical approach- 
as to NAEP data are neither atralghtforward nor Inexpen- 
alva. The apaolflo problems this situation poses for 
oartaln analytloal approaohas are described' In detail In 
Issue Paper 97. Here our attempt Is only briefly to 
dasorlbe analytloal studlaa that could enhance under- 
atandlng of equity issues. They are of two typest cor- 
relational analyaaa and general linear model analyses. 



112 



CorreXttloneX analyses * Although NAEP's data set has 
a ooiDplioated ^ Incomplete ooverlanoe structure , It Is 
nonetheleaa possible to examine relationships of re- 
sponses to various seta of exercises emong and between 
verlous groups of people Ce«g.^ women, blacks, any group 
sufficiently sampled)* NAEP staff have conducted some 
limited principal component analyses and csnonlcal cor- 
relative enalyses Ce«g«, Knight and Johnson, 1978), but 
not with regard to any equity Issues* Most recently, 
Jene Armstrong conducted a correlational study of an 
Important equity issue (Armstrong^ 1980) . Using NAEP 
date m part^ Armstrong analyzed the Influence upon 
women Vs mathematics partlclpstlon and performance of 
numerous factors such aa sex-role stereotyping, cereer 
plans , attitudes toward mathematics ^ parental Influence 
aiid socioeconomic status* However , Armstrong* s study 
was a special effort employing a speclel sample, admin- 
istration and questionnaire* It was not based upon NAEP 
data previously collected In the usual way* 

General linear model analyses * Assessment results 
estimate relative levels of achievement for subpopula- 
tlons Such as blacks or people : living In the Southeaat* 
Interpretations of these group percentages, however^ can 
be misleading m several ways* The label **Southeest^ 
should not be taken to mean that performance Is solely 
the result of the fact that respondents live In the 
Southeast* A large fraction of respondents In the 
Southeast happen to live In rural areas* Conaequently^ 
slze-and^type-of-communlty effects may appear to be re- 
gional effects* Similarly, persons whose parents went 
beyond high school are more numerous In affluent commu- 
nities than In the country as a whole, and peraons whose 
parents had no high school are more numeroua In rural 
communities* In this case^ parental-education effects 
may be masquerading as slze^and-type-of-communlty ef- 
fecta* 

Confusion about group effects arises when the mix- 
ture of. characteristics Is unbalanced from one group to 
another* NAEP*s weighted probability sample automati- 
cally preserves this Imbalance in its percentage esti- 
mates. **Balanclng** Is e linear model technique that 
simultaneously adjusts a set of subgroup proportions to 
the national average (Tukey, 1970; Larson et al, 1973; 
Larson and Searls, 1974). When results for blacks are 
balanced to account for the fact that disproportionate 
numbers of them reside in the Southeast^ live in Inner- 
city environments and are m less than modal grades, the 
black deficit Is usually cut in half* That is, if 
blacks* unbalanced difference from the national level of 
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pepfopnanoe la 16 polnta, the balanced pepcentage is 
uauaXXy apound 8 polnta. Thla Is just anothep way of 
saying that poughXy half the blacks' dlffepence can be 
aooounted fop by the pathep gposs measupes of such fac- 
tops InoXuded in the aaseaanent as pesldence, living 
pattepna and home enviponnent. 

The National Aaseaament ahouXd pepopt mope balanced 
peaulta than It now doea, but two consldepatlons should 
be kept in mln^. Flpat, balanced pesuXts ape hapd to 
Intepppet* NAEP doea not oXalm that such vapiabXes as 
"Xow matpo" ape vepy ppeolae, but when It "subtpacts" 
Xow metpo fpon bXaoka' pepfopnanoe it gives the appeap- 
ande that It haa ao'oounted fop aXX of the detplmentaX 
effeota of Xlvlng in an Impoveplshed upban envlironment. 
when aevepaX of these faotopa ape accounted fop and 
bXaok/whltea' dlffepenoea renaln, what infepences can be 
dpawn? That the PealduaX dlffepence Is school caused? 
That It peppesents dlffeplng attitudes towapd tests? 
BaXanolng oXaplflaa aoae aapeots of the equity ppobXem 
but It palsea new queatlona at the same time. 

The aeoond ponaldepatlon is that whlXe baXanclng 
dpanatlzea the oonplex Intepaotlons that influence dlf- 
fepentlaX aoedemlo pepfopnanoe t it does not peflect Pe- 
allty. Balanolng aaka what if blacks appeap in NAEP's 
vaplabXe oategoples in the same ppopoptlon that a na- 
tlonaX popuXatlon does? But bXacks do not appeap in 
these ppopoptlons. nop Is It necessaplXy tpue op even 
deslpabXe that they n\ist do so. 

BaXanolng la one exampXe of a muXtlple pegpesslon 
anaXyala teohnlque applloabXe to the NAEP data base. 
Thepe ape othepa. MuXXla <1979) adopted CoXeman's 
<1975) pegpeaslon and path anaXysls ppocedupes to cltl- 
zenahlpt aoolal studies and mathematics diata coXXected 
In 1975-76 to eatlnate pelatlve school effects fop 17- 
yeap-oXdfl. Noe (1978) employed a slmllap apppoach to - 
study the peXatlonshlp between oeptaln attitudes towapd 
Solenoe and pepfopnanoe on oeptaln science assessment 
Items. No equlty-peXated studies of a slmlXap kind have 
yet been undeptaken , but thepe Is no peason why they 
oannot be, whenevep sampXe size pepmlts. Reppesentatlon 
of equity gpoups in the NAEP data pemalns one of the 

most opltloaX faotopa that Xlmlt analytlcaX wopk of this 
kind . 
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PART 3 

NEW DIRECTIONS FOR NAEPt 
THE TRADE OFFS 



The following table (Table 4) lists suggestions for 
changes In NAEP to Ipppove its oontrlbutlons to our un- 
derstanding of the problems of literacy and equity. 
Most of the suggestions have been discussed In the pre- 
ceding pages; others are mentioned to stimulate further 
thought. The table should be seen mors ss an illustra- 
tion of the kinds of thinking that have to be done tKfem. 
as a definitive statement about probable consequences of 
changes In the Assessment. 
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TABU 4f Clia^^ tha National Assessment and Potential Trade Offs 



Suggested Chany 



Closer llnka^ to 
fitthnographtc studies 
Qf literacy 



PoiilbU IHtMn 
■ HtralMny 



Analysis of causes 
of Illiteracy and 
ii^rovenents In 
literacy 



HO 



Progran evaluation 



Crudely 



CtunoH In Model 

Nbrt bKl^und vaHa - 

Met leit coit 

Add Inttrvlem atdlun 

coit 

Add tilt itudlet , obier- 
vatloA — high cost 



^ Hike HAEP wrt mt ex- 
porlMntal ttudty -- 
lOMf cost (If descr1|H 
tin nliilon abindwied) 
CollKt icbDOl. cltii* 
rooi ind fiiMly data 
not colloctabit with 
currtfit coiiitrtlnti — 
■idluii cost : 

filter «ort txict pro- 
Sfii Inftmatlofl 
Mdlw coat 
Change impllno proee- 
durt 'ftf ftchooii and 
studinti Hlthln 
ftchooii — high coit 



Trade Offs 



* Sicliground wirlables cut 
dow exim . 

V InUrvlens cut down exer- 
cli :tlMe^X1ncreose scor-' 
Ifig co$£s;:\scor1e9 tin 

* ObifrratloMV studies > r 
cMi»9e chancteir of M 
could 1hniMnce|»iiit1c1^ 
tlO|irTatd;\l^ 

t Itadlcal change In fimctim 
iMHkH radlcall/ alter audl- 
tnce Interest ihd use < 

* Probably Violates teglsla- 
tlve Intent 



* HAEP becomes More contro* 
verslaV aiid.si^ject to 
pretiure-group tattles 

* Loii of sose lorijgjvter* 
reltVance. ilnce prograis 
change andifieW sitwIe could 
be less useful for long 
tart . 

* Could lafUct partlcliMitlon 

Bta adversely 
thout extra finding, 
cuts Into assessment t1«e 
and resources 
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Problews 



Coordination 
. Turn aroimd t1«e 
Integration of very differ- 
ent JilNs^ of Infonutloii ^^f: 
Conc<^~tua1 Incongruity be-;-'^ 
tween NAEP and etlinograplq^^ 
could 1^ to less coherent' 



* New staff, new orlenu- ' 
tlofi. iiGM politics :^ 

* Hass1vft technical probleits, 
infer^tce prbblettSt etc.; 

* Probably cahbot be done :v 
iiell t since auch Influence 
comes froM outside the '^^^ 

, schools.. 

. - - 

' '.. .'"^ 

* Many of the sane prob1eias_ :' 
Title revaluations faced/-; 

* Precision, validity of data! 
relevance of results to any^^ 
particular school could be^"' 
questionable 



ERIC 



I ■ 

fe^ SwKWSted Change 

4;^^ Craitt literacy 
it-.-- asitiiiMnt 



P05sib1e Within 
present Httdel? 

Ves 



Surv^r adult literacy Ves 



TABLE 4 Continued 



Chanoei in Model 
or Addttloni » Cflit 

• Use eNiiting r^dtng, 
nath, literature; 
iiritifig. conaumr 
skills, cit/ioc tNer- 
cisei «' low coit 

• ttpan^ at sesfwnt 
ti«e so that one itu-- 
<tent could take all 
eNerclies - lotf coit 

« Add i4u1t aiseiiwfit - 
medim colt 

• Create new area with 
its own objectlvei, 
itfiott. etc* low 
€vst if U replacH 
current area, hfgh 
cost If U is addi- 
tional 

« $500,000 for United 
assessoient 



Tr^dc Of f s 



* Lose those Items for other 
assesswnts or change re- 
lease policy 

« Lose advantages of matrlN 
sanple 

* ProMbly lose soioe other 
subject(s) (e.9.. MHJsIc 
and art 

* Expanding these could 
create school participa- 
tion problens 



Uithoiit extra funding it 
would drain resources fro« 
other age levels 



Prohlews 

About the same prohlens en 
countered in creating s^y 
area 



None that have not boon 
dealt with In the pest 



Survty literacy amng 
American Indians and 
other groups 



Marglitalty 



Larger siopla or over- 
sutpling of groups 
«ed1i»nigh cost 



' With llttltod resources, 
trade offs iiKlude lowering 
muter of pacl^ages losing 
national precision 
Main trade off of larger 
sanple is Mney 
Naad new hackground vari- 
ables wre relevant to 
Indien education 



None that have not been 
dealt with in the past 



''A 



0:' 



a Suqaestcd tf^noe 

Exftrcfses cross* 
Indexed to facni- 
otate "roll your 
own* tftariry 



Experiment utth nm 
; tftmcy M«uii^ 



t^* -' Broiden M£f coveriQft 



Ov: Expand cross cliss- 

ff fcitfon. cross- 
r Ubtilitlon of tW£P 
^j^.;: Variables. and results 
^V:': in order , to report * 

eqifftlfty 
it;' r'of educatforul out^ 

cOMi and changes 
f^v; for polfcy^rtlevint 
groups . 

Invest f{iat« tquiHty 
M of iccess to ichool 

: resources . eqtial f 
^^^=e;of opportunity to 
f ' tJice Bdventege of 
i^i ..retources 



Poiilble Wfthfn 
Present Hodel? 

Yet 



See pp. 22-24 
Yil 



Soimttat 



TABLE 4 " Continued 



Ctiengei H Hodel 
or AddUlofls ** Cott 



See p. 
Ion 
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** colt 



Add suMect Areas te re- 
flect dfvarsl^ of 
school of fei*1iigs — 
Mdfi» to hf^ colt 
Perfonq iliaSsnents for 
other 9inoups at thefr 
expanse 

Sofse netf programing 
loH cost' 



Hequfres wich wre thor* 
ough descrfptlon of school 
resourcesp daflnltfon of 
access. ^usHtatlv* fnfor^ 
Mtfon fn addftlon to 
quantftatfve ^* htgb cost 



Trade Offs 



ftew prograwlrtg 
Potentfdl abuse (e*g.i 
creation of coftpeteitcy 
tests frop tton not 
created for coMpetency 
testlogi 



Breadth could H purchased 
at soM expenses to depth 
ftroader audtencet expanded 
relevance 



• Lengthens analysts and re- 
porting tljoe 

• CoMpetes for natnltne and 
research ttoe 



■ Uoutd covproselse annint of 
tlM dedicated to gather* 
Ing performance data 
unless new resources added 



Problc 



* Defining classification v 

• Attempts to do this In thi 
past have ancotmtered i^:^ 
thorny conceptual and - 
logistical problens In di^< 
scribing exercises and'V^ 
Making then readily avail- 
able 1^ 



Money 



Nona that have not been 
dealt with In the past 



Many of the problems that ^^^ 
arose fn title I and FoIIm 
Through Studies e.g*^ liu 
tiple Interpretation of ac*^ 
cess. HNiltlple definitions! 
of resources, the polltlcs^^ 
of a decentralized aduca-ii^^ 
tlondt system 



Investigate effects 
f-\^ . of coMpinidtory ed- 
ucation prosrams 



TABLE 4 " Continued 



13. Investigate chanQiS 
In racist, sexist 
barriers to equal 

... opportunity and ef- 
fects upon achieve* 
nent 

H; l^cribe changes In 
achlevoMnt In titias 
of h1gl)-i JiedluBH 
an<1 loH-achlevanent 
prmips to see If 
they are perfonatnQ 
differently 



Possible ilithtn 
Present Model? 

Ho 



Ho 



YiS 



Changes In'^Hodal 
or Additions Cost 

Requires experlnental 
approach, different sam- 
pie> Could be done as 
a special probe at ad- 
dltloMl cost, but best 
dona as swil-scsle 
stu4/^ Hf£? could add 
background variables 
relatlnp to conpense^ 
tory education 

ttequlriff Identification 
of racist, sexist bar* 
Hen detectable t^tth 
quantitative Mflsures 
or special observa- 
tlonal probe 



Ko change 
cost 



little 



lE jide Of 11 



Probli 



• Unless organized and 
financed separately, would 
coMpronlse descriptive 
data gatliering 



Even If done veil would be 
controversial, possibly 
affecting ypluntery co- 
operation rate and Integ- 
rity of the sanpli 



• Takes spotlight off minor- 
ity groups and turns It 
Instead upon achievement 
groups 



Sane as above 



Substsntlal neasureinent 
and political probleois 



Kone 



APPENDIX A 



TRENDS FOR TWO KINDS OF LITERACY 



Following 13 a discussion of the two literacies 
defined In Part 1 . 



Literacy A 

1 • Once some problems accessing NAEP Information 
have been dealt wlth^ group A will find that over 90S of 
the nation's 17-year-olds can perform '•funotlonal'* read- 
ing tasks (Functional Literacy ... , 1976) . Exercises 
assessing ^literal** oomprehenslon skills In the 1970 and 
1975 reading assessments Indicate that the vast majority 
of teenagers can read at this level and the percentage 
Is not declining as some education critics have oleimed 
( Reading Change, 1^70-75; Summary Volume , 1978) . Stu- 
dents from rural areas and small towns are Improving as 
are those In the East South Central region (Alabamat 
Kentucky, Mississippi and Tennessee). Southeastern stu- 
dents In predominantly black schools appear to do worse 
than those In schools with 20 to 90% white populations. 
In all studies, girls perform better than boys; students 
from homes In which a parent had post high school educa- 
tion perform better than those from homes In which nel-> 
ther parent attended high school; students from hlgh-SES 
districts performed better than those from low-SES dis- 
tricts; and 17-year-olds In the 10th grade performed 
considerably behind (13 points) the age group as a wh- 
ole . 

2. National Assessment has not tested knowledge of 
grammar and grammatical facts, so group A may be dlsap*^ 
pointed on that score. It may also find NAEP^s prefer- 
ence for descriptive, rather than prescriptive, treat- 
ment of writing difficult to adjust to. ^We're not In* 
terested In the characteristics of the essaySt^ they are 
likely to say, *^Wi Just want to know how good they 
are." Quality Is, in fact, defined for many writing as- 
signments, but the job cf aggregating all the per- 
centages from task to task Is left to the reader, and It 
can be bewildering . Only 17$ of the teenagers appear 
able to write a satisfactory job application, but 54$ 
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can write a eompatant essay about a picture. What does 
this mean? 



NAEP's report, Writing Meohanloa ( 1972) , provides 
aoae data about Literacy A. The i7-year-oXds made be- 
tween two and three punctuation errors per 100 words; 
about one sentence fragment , run**on sentence and agree- 
ment error per essay; and about two mlaspeXXlngs per 100 
words. The error rates were highest, of course^ for the 
worst writers. Inflating these figures. Two- third a of 
the average and good writers had no capitalization er* 
rors , three** fourths had no sentence fragments or run-on 
sentences and over 90% made no paragraphing errors . 
These are not particularly alarming figures. And , al- 
though the sample essays Included In the report do not 
Inaplre excitement about basic writing ability, neither 
do they lend support to the belief that a great many 
17-year-olds cannot write a simple sentence. The 
telephone*book sized report Selected Essays and Letters 
(1972) , a collection of almost 10,000 writing samples 
from the first assessment, presents writing Information 
In the most concrete way possible. Since It provides 
data about sex, race, community and parental education 
for each essay. It permits any number of analyses. 
Group A would probably feel that the majority of teen- 
agers Is doing relatively well with mechanics, but It 
would probably be stunned by the problems that appear In 
essays Judged to be of low quality (the bottom third of 
the distribution). Huch work to be done here. 

Writing Mechanics, 1969-7^1 (1975) does not directly 
address the concerns of group A as much as the earlier 
writing mechanics report does. It does, however , com- 
pare writing from two different assessments and finds 
the more recent writing lower In quality. Surface 
structure characteristics of the writing do not appear 
to have contributed much to the decline, so It Is safe 
to say that the relative occurrence of misspellings and 
errors of punctuation, ' capitalization, word choice and 
the like remains about the spme. What the results did 
suggest would be of greater Interest to group B, for 
there are Indications that the decline In quality was 
due to deep structure problems such as Incoherence , the 
amount of modification taking place and (a problem Iden- 
tified In the earlier report) understanding of the con- 
ventions of written English. 

3. Hath Fundamentals , a 1975 report on the first 
mathematics assessment ^ establishes that well over 90% 
of the 17*year-olds could perform simple addition and 
about 85-90$ could perform simple subtraction. Between 
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80 and 90% oouXd multiply and divide, although the per^ 
oentage usually dropped when decimals were Involved . 
Simple computation with fractions and Integers proved 
more difficult , but percentages were In the 70-dOS 
range f nevertheless^* Whenever computation Involved 
translation (e»g» » word problems) , the results were 
lower for different groupe* 

The recently reported results from the second math- 
ematics assessment do not reveal a marked decline In 
these fundamentals*, Although there was an overall S% 
decline In skills^ most of It was caused by declines In 
algebraic and fraction computations, not In simple addl* 
tlon , subtraction , mul tlpl Icatlon and division * Liter- 
acy A may not be advancing, but neither is It losing 
ground* There are still marked group differences worthy 
of attention, however* Although younger blacks seem to 
be closing the gap somewhat, the 17-year*olds remain 
considerably below the nation* 

^t* Bloom* 3 Taxonomy of Eduoatlonal Objectives 
Cognitive Domain (197«) distinguished several levels of 
cognitive ability, ranging from knowledge ( the lowest) 
to comprehension, application , analysle, synthesis and 
evaluation ( the highest) * When science questions were 
classified according to the cognitive abilities required 
to answer them, the result Is the pattern of group dif- 
ferences shown In Table A-1 * 



TABLE A*l. Group Dffferences From National Level 
of Science Performance by Cognitive Level 

Knowledge Comprehen sion AppMcation Analysis and 

Synthesis 



Whites 


+ 2.2 


+ 2.4 


+ 2.8 


+ 3.3 


Blacks 


-13.4 


-14.4 


-17.0 


-19.6 


Hispanos 

No nigh school 


- 9.5 


-10.2 


-11.5 


-12.3 


- 7.4 


- 7.3 


- 8.7 


- 9.4 


Post high school 


+ 4.6 


+ 4.9 


+ 5.3 


+ 5.7 


Low metpo 


-11.8 


-11.3 


-13.0 


-14.7 


High metre 


+ 4.6 


+ 4.4 


+ 4.2 


+ 5.5 


Big city 


- 5.1 


- 5.4 


- 6.2 


- 6.8 
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If we presume that Literacy A is defined more In 
terms of knowledge and oomprehenslon-XeveX skills <wlth 
some appXloatlon) , then there are considerable differ- 
ences at even the most basic level. More worrisome to 
those Interested In Literacy A, however, is that differ- 
ences become greater as one moves from left to right, 
I.e., up the skill ladder. It becomes clear that even 
Literacy A has some components that are more troublesome 
than others. And while watching the gap between the 
"haves" and the "have nots" widen with the difficulty of 
the task, some Literacy A proponents may take a greater 
Interest In Literacy B, where differences appear to be 
most extreme. 



Literacy B 

1. The results for hlgher-XeveX comprehension 
sklXXs are Xower than they are for Xower-XeveX sklXXs. 
John MelXon <1975) transformed each HAEP Inference Item 
Into a decXaratlve statement the students had to com- 
pXete by choosing the correct answer. In doing so, he 
found that the more words per T-unlt <0aln cXause or 
sentence) there were In a statement, the Xower the per- 
centages of success on the Inference Items. For In- 
stance, the average percent correct for statements with 

13 words per T-unlt was 91.6%; 14.3 words per T-unlt, 
79.4%; and 17. 3 words per T-unlt, 56.2%. Thus, Infer- 
ence resuXts are confounded with syntactic compXexlty, 
vocabuXary and other aspects of the question asked, and 
group B wouXd be weXX advised to define Its Xlteracy 
more preclseXy. Fewer l7-year-oXds show sklXX In com- 
prehending reading naterlaXs that require a certain 
amount of Inferenclng » for exampXe, recognizing that 
an author is making contradictory cXalms or inferring 
from a teXevlslon scheduXe that If one Is watching pro- 
gram A he cannot be watching program b. Since many 
"functlonaX" eadlng mater laXs — warranties, guaran- 
tees. Insurance poXlcleSf Xoan agreements, etc. — re- 
quire compXlcated Inferences , these Xower percentages 
bear dlrectXy on the practlcaX effects of IXXlteracy. 

As was the case with Literacy A, advantaged groups 
tend to perform better than disadvantaged. But for Lit- 
eracy B, the gaps are even wider. The bXack/whltes* 
difference for aXX reading exercises, for Instance, is 

14 points; for the Inference exercises, it Is 24 points. 
The gaps between the no-hlgh-sohooX and post-hlgh-schocX 
groups and the Xow-metro and high-metro groups are aXso 
Xarger . 
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Although, as noted earlier, low-XeveX reading 
skills do not appear to be deollnlng, higher-level 
skills are. The deollne of 17-year-olds' reading per- 
formance between 1970 and 1975 Is almost entirely at- 
tributable to lower percentages of success on Inferen- 
tial comprehension Items. This Is some support for the 
contention that the two literacies are different in 
character and probably require different policies and 
strategies. 

2. Group B defines writing literacy in terms of 
ability to master basic syntactic and rhetorical conven- 
tions. As a minimum the group would expect 17-year-clds 
to be able to write coherently in a number of different 
situations (e.g., school, work, family) and a number of 
different modes ( letter to the editor , letter to a 
friend, simple bock report. Interoffice memo, etc.). 

The first writing mechanics study (1972) seemed to 
show that most papers written by 17-year-olds shewed 
"mastery of basics." Nevertheless, there was a consider- 
able difference between the "middle quality" and the 
"high quality" papers, consisting prlmsrlly in the bet- 
ter writers' apparent ability to choose among a number 
of various constructions Instead of being limited to few 
or no choices. 

The second writing assessment suggested that the 
factors that contributed most to the lower quality of 
17-year-olds' writing were not surface structure fea- 
tures such as spelling, capitalization, and the like. 
Rather, the decline was most probably caused by in- 
creases In the number of run-on sentences and awkward 
constructions, coupled with n^rreases in the use of oom- 
plex sentences, racdlfloatlcr ..nd conventions critical to 
maintaining coherence. The second assessment also 
shewed considerable variation In the percentages of stu- 
dents able to write for different purposes and audi- 
ences. This suggests that s good many teenagers proba- 
bly do not have flexible writing skills that can be var- 
ied as situations dictate. 

As In reading, then, Literacy B in writing Is de- 
clining. It is Important to note that the deollne Is 
entirely accounted for by middle- and low-quality writ- 
ers. There appear to be as many excellent writers as 
there used to be. Thus, the declines In Literacy B are 
not declines In advance or esoteric skills,, useful only 
to elite writers. They are declines that affect the 
average and poor writers who constitute the vast majcr- 
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3. Recent Congressional testimony has succinctly 
summed up the trends in mathematics* Edward Esty: 
"One, most children can do simple whole number computa- 
tions accurately; two, there are weaknesses in more dif- 
ficult computations with fractions, decimals and per- 
cents; three, children have more difficulty in applying 
computational skills than they do with the skills thems- 
elves; and four, there are weaknesses in higher- lev el 
skills" (Subcommittee, 1979). Shirley Hill: "The ines- 
capable conclusion to be derived from the results of the 
second national assessment of mathematics is that there 
is a critical need for attention to higher-order cogni- 
tive skills* Reasoning, analyzing, estimating, select- 
ing appropriate information and inferring — these are 
basic skills that are essential to the effective appli- 
cation of mathematics" (Subcommittee, 1979)* 

* The third science assessment indicated that 
17-year-olds' level of performance continues to decline. 
-However, it is declining fastest in the area of physical 
science. The majority of questions asked about physical 
science involve application, analysis, synthesis and 
evaluation skills* In biology, which showed a slight 
slowing of the drop, the majority of questions involve 
knowledge and comprehension * Thus , there are grounds 
for suspecting that the science results reflect the same 
decline in Literacy B skills that the other assessments 
show* And, as Table A-1 reveals, group differences are 
greatest at these higher levels* 
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APPENDIX B 

A SAMPLING OF EQUITY 
GRAPHSt TABLES AND STATEMENTS 
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2. 



White Attitudes toward BUcks 



Statment 


\?6Z 


\97\ 


\97B 


Stacks have less Ambition than whites 


66 




4? 


Blacks ate more violeni than whites 


NA 


J« 


34 


fUcks want lo Jive offihe handour 


41 


%9 


3« 


BUcks'have tess nflttve intetligenfe 


J? 


J7 


15 


stacks breed crime 


35 


17 


1? 


Blacks C4itf 1^s& fot the family thaa 








whjiet 


3> 


16 


tS 


BUcks ate inferior lO while people 


J" 




15 



Source; The Naiional Conference o^ Christians and Jews^ *'A Study of 
Anhudes loward Racial and Religious Minotiiies and toward 
Women/* Novembet i?78, p. 16. Conducted by Louis Hatris and As* 
tfoctaies. 



From: Dom, E. Rules and Racial EQuaHty > p. 51. 



3, Percentage of PopuUrion 15 to 34 Ycari Old Who 

Completed 4 Yeats of College or Morc» by Race and Sex 







Black 












Total 




F^maU 


Tortf/ 








4-1 


4.1 


4>0 










5- 




6pI 


I j.^ 


iS,? 


10.4 


t?70 


6.1 




6^4 




to.? 


ft « 


l?74 


8.1 






ii.c 


U*9 


I7pt 



From: Dorn, E. Rules and Racial Equality , p. 40. 
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4. 



Ywt ct Rfigular Schoollr^g Compltttd by Dtffersnt Popul»tior\ Groups 





Mtin 


Otvlition 


Coftffie*'«nt 

of V4lt«tiM 




All lmtIyi<Jua]s 










Bemt 189S-t904 


&90 


i76 


a42 


\ U) 


190S-1914 




3.63 


037 


(105) 


191S-1924 


10.86 


uo 


030 


(112) 


1925-1934 


1L47 


3^1 


028 


(113) 


19 1939 




2.92 


0UZ5 




1940*1944 




180 


023 




Mtltt 










Sofnt 189S-1904 


8.77 


3J9 


0A4 


( 8-7) 


1940-1944 


1Z39 


3.00 


024 




F«milcf 










Bomi 13SS-1904 


a96 


165 


0t4l 


( &9) 


1940-1944 




t57 


021 


(123) 


Whitii 










8omi 169S-1904 


9.18 


165 


040 


( &» 


1940-1944 


1^31 


177 


022 




filtelts 










Bemi 1895-1904 


191 


176 


054 


( S.1) 


1940-1944 


ILIO 


177 


025 


U2Jt} 



Source: Rowi KJ4 weft dtfHy«d by Nonru Riines for CEPft from 
Burvtu of ttic Ccfuu^ *'£dueit(onal Atuinm^nt in 1969r Table 1. In nteuitlini 
mttpt wad itandard dcviatfof^ii inrt»yidMi)i reported ts haytn); 0 ta 4 ytftn oC 
kIiooI ailocftied folkwi: 7$ peretnt to 0 yeatf. 25 rcfctnt (0 1.5 ytafv* 
SO ptmni 10 3J yetn. ]nUiyiitu«U rtportlnt S or inorc yeatf oC eollfsA wm 
titoctttd as rollout; 50 pentnt (o 17 ytan^ £5 percent to IS ye^rji* 35 Pirftnt (o 
19 yttfi^ Preschoolmg it exclitdtd^ B«vtriy Duncan obtained fnetEontilly lowir 
BMni usint stiChtJv d:ffe:«nt JUiumpdom <ieo her *Titnds in liM Ouiptit 
Diitribuiton of SdiooU&c**). 



From: JenckSi C. et al* Inequality! A Reassessment of the Effects 
of Family and Schooling in wnerica , p> zT* 



Percent ot degrees awarded to lemalefli hy level o[ degveet School years 
ending 196S, 1970, and 1977 



degree 


1969 


1970 


1977 






Percent 




Badtefot's 




43.1 


45.1 






39t7 


47.1 




10. a 


133 


Z43 


Fifst'professionalr.. 


3.^ 


5.0 


18.7 



SOURCE US. Dipiftmint ot Hiaith. Edueation. and 
WeJfafo. r4atlonil Ctntar for Education Statisticit 



From: The Condition of Educa-tlon. 1979 Edition ^ p* 230. 
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Pemot of persons 25 years oM and over completiog at 1^ 4 yeais of high school, by age 
and facial/ ethnic group: 1977 



Percent by group 



Radal/tthnic 




25 


30 


35 


i5 


55 


65 




Totai 


to 29 


to 34 


to 44 


to 54 


to 64 


afi<l Ofver 


Total 


64.9 


asj 


BLD 


73.6 


64.3 


56.2 


37.5 


Whit* 


67.0 


S6.8 


az.6 


75.6 


67.5 


59.3 


39,7 


Black 


45 J 


74.5 


67.2 


59.7 


35.6 


26.1 


14.8 


Hisptnic ongm ^ 


39.6 


58.1 


49.D 


41,0 


33,0 


22.7 


16.2 


Male 


65.6 


S6.6 


az.4 


74.3 




55.5 


36.0 


Wtiit* 


67.5 


67.6 




76,2 


66,7 


53.4 


38.0 


Black 


45.6 


77,5 


69,3 


55.7 


31.1 


25.5 


15.8 


Htsptnic origin V,, 


42. 3 


614 


S3.6 


43.7 


35.7 


2D. 6 


19.4 




64.4 


S4.2 


79.7 


73,D 


66,0 


56.8 


3ft, 5 


White.... 


66.5 


86.0 


nj 


7S,3 


69,2 


60,0 


40.9 


Black 


45.4 




65J 


55,7 


39.2 


26,6 


14.1 


Hispwie OfiginV.. 


37.2 




45. £ 


3S.D 


30.S 


24.1 


as 



' Categorfee are not dlscroie (e.g^, a peraon mav be cJaaa^fiod rn bo(h whita and HisPanic 
origin catagorfas). 

$OURCS; U.S. Oapartmant of Commtrct. Buraiu the Cansua^ 9^^catlon$i Att9>nm§nt 
in tha UniM Statw Kfvch f 077 and im, Saflaa ^20. NO. 314, i9r7. 



From: The Condition of Education, 1979 Edition , p. 224, 
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'<t Ml '#J 'M '** '*T '« '7<* '71 '7* >J 74 '71 '7* "77 

From: Dorn, E. Rules and Racial EgualitY t p. 43. 



8, 



Eqaa^izafiofi of Edacction Hat Set Bc4n 
Associated with Equaiization of Inccmw* 




1974 



YCAA 



fMttiT Tit* iippn Hm them tM trcMf ont tint* In (h« d«frn 
Af JiKqij^itf 0f jflcofM* ii mtuured the ilVidafd tfirtttwa 
«f tht ntiutaJ (otafithm or umuat incom* ft7 fKikn •tvd 
twtnitiAtc or «lur Titu lowff Um tb^*t the tffttd OHf Un* in 

WflicJmt of v«*j>t>M fiiniUrd dcvi^tioj d^tldtd by tbs 

ud oidiEt. Ouifw 1)70 10 lf^4 iftctuiiutfAbtMd m 
C«a*u4 

Ch4nv«* in P«rMnat IflAHM IncqiMlitr In itt« U^S^"* Jpumti 
fit PmUtctA ffomF^^i V«l. 10. No. 3, Pirt t| (Ma^iuM l*n>. 

From: Bowles. S. and H- Gintls. Schooling In Capitalist America ^ 
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9. 



Mediu 

















QonstMt 
















Doilsr 








DifffTtttta 












"9 Jo 


*^ ^ J 






J4 


I*)IT 


4771 


i$9i 


11 SC 




J>J9 


»o)i 


iliT 


*n 


.1*11$ 


49$ 9 


iCii 


l)4l 




4M4 




ITT* 


*JT 


i*ifi 


1171 


1941 


ll$4 




4J9l 








i.M' 


$4ll 


)07i 


X4tO 


I9f4 


4n9 


14 lO 


If 19 




l*M« 


3 J 19 


*993 




'9JJ 


4«C»$ 


1J49 






l*MT 


$741 


3i7l 


lf<4 


i9y* 


499J 








1*119 


<nc 


31)0 


190^ 


I9JT 




lT*4 


MAI 


*J4 


l*ll< 


<ii7 


P7I 


ll49 


I9$t 


JJOO 




M<f 




MH 


Cm 


)|J" 


1991 


"fJ^ 




Tfof 


«l4< 


♦J* 


I.l4f 


<74l 


)4f9 


U$9 


[9^ 


J'JJ 




I«Ol 


*3f 


r*liT 


•$7« 


3«4) 


19» 


i9«l 


J9»i 


)i9t 


*Tf« 




l*ll< 


«7) 


)$•" 


)""4 






mo 


*W 




l*l<M 


Cite 


3«7< 


JIIO 


'9*J 


<f4l 


J4*J 






l*Otl 


7144 


)7t* 


»«4 


'9i4 




3TM 


JIJ4 


*J4 


1*97^ 


7>79 


4007 


)371 



i9#$ 


7*$* 


uu 




*$4 


1*0$ 1 


T#T* 


4111 


)$« 


I94C 


7791 


4 $07 


ntj 


*M 


I*0i9 


toil 


4«)t 


33^0 


19*7 


■^J4 


4l7$ 


JH9 


■19 


1*000 


iL)4 


4lT$ 


)3$9 


I94t 


«93T 


$)<o 


*$77 




.9«0 


iFlO 


$<4* 


)4$4 


i9<9 


9Tl4 


$999 


170 


*<l 


.911 


■91) 


$4<$ 


)44l 


i97« 


toi$* 


«i79 


)9$7 


*<l 


*l«o 




$400 


)40) 


1971 


lo*T* 


*44Q 


41U 




J14 


IT94 




34ft 


1971 


ii$49 


«S«4 


4"$ 


*$9 


nT99 


9Ul 


$414 


)T44 


I9T) 


ii$9$ 


7i<9 


$u< 


*yi 


*Tli 


94TI 


$4** 


400$ 


I9T4 


i3H« 


;laS 


n4i 




*4Tl 


ff«I$ 


$1V4 


37* i 


197$ 


14164 


<779 


^■9 


*<i 


.dil 


lUO 


$4tl 


340* 


I9T(S 


M$3r 


9141 


<M9 


19 


*$IT 


9rio 


$T)9 


)3ii 


1977 


I674Q 


*$*J 


7177 


*yT 


*n4 


9iT4 


$iyf 


)V7* 



* ■ ^o"?"^ ^■'L*'A*^ "'^^"^ Popwiiliofl of rhf UAii«U Srmn, 1974"* (W^Jhinpotii CPO* 197 0, table 
?iIf w .'^"^ ^***7 l'nmil*M and rufiiAi jn th* Untied Statrt, » 977** 

tW.nhinewH*: CPO* )97S); I 10 ihiAk DwJ^i Johnion of th* Ccnioi Sor^u tor helping mc l<«iMhrmc 



Wtfimn Burrau mestund hUck wcomc Mptrilflf* 

AbiitMti of ^ Umttdium, \m, Tibb 711. p. 47* Cuftrti jiwomt hmn CPPI «qMtt cumtiw w- 



From: Dorn, Rules and Racial Equality , pp. 34, 35, 
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12. 

A^erjgtt unnuul ^ luluricA ot 1967*77 bachclfir*! und mtuter*i icfitee reciptents worlUog (uII«Hrw, by sex 
and by niciiil/cttime group: Kbruur}- 1V78 



Chiraderisttc 



Avtrid innual salary 



difiH 

rtciwintt 



nci{>i«nti 



Totar $1U00 $16,000 

Wmte, non^Hlspanic. . . 11,700 1S,S00 

Black non-HispinicH . . 11,100 17|C0O 

OUier> UiSOO 16,300 

Mar* 12,700 16,900 

Whit«, nofl-Hispanic. . . U, 700 17, OOO 

8tai^, non-Hisfnnic. . . 12,000 17,200 

OUier^ 14,500 16,100 

Femal§ t0.300 14,900 

WhiK nm-Hispanic. . . 10,300 14,650 

BiKk non^Hispanic. . . 10*300 16,800 

OU)er> 11,000 16,600 



SQUnCE: Oeparlmsni of HsaJth, Education, and W«U 
tarCi Nstlonal Con^tff (or EducsMon Stttiincs. I'^TB Sttfvty 
d/ 1976-77 CotW0 G/idut(ttr prslimtnarv data* 



From; The Condition of Education^ 1979 Edition , p, 240. 
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13. 



Education^ A it^mmnts Art Strongly D*P<^ttdw on Socio! 
Background £wa f^r Ftopie of Similar CMdhocd LQj 
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14 



t3 



12 



13.7 



13 



11 .ft 




1212 

I 



i2.a ^ 

I 

I 

i. 



13.4 



I 

I 
i 



cMtt «itlint!'i4 imM* mwiber of r«m ol Mbw^ipf 
iiulM4 br til n»n from that ctoup* Tlw i^lhfMund bu lodl^ 

uukMd br vlth IQ c^atl 10 Iht «v«fais for tU 
cmtft fAmpli^ Ihi lintpte fffcn '*Tion-N4>re'* men of "imwh 

•oum: Simvit ftovlu ud v«ltri« NitMif "Olw *lp)wif* 
ttoci of 10/ 4it f itii ^nittri«Mr«<ioAil Trailimifiion cf Eoo- 
voaric lBnmntH7f ' Tkf Itcrffv «| rcoi*oiiTi>« «nd Stttttrtet* 
Vol LVl, N«4 I' 'Mnurr It«pf4nltd ^ pilAtuud of th« 
Ftcddcm and F^tl j»l of Ff t^^sf d CMi<P< 
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From: 



Bowl as, 
p. 31. 



S. and H. Sintls. Schooling in Capitalist AmeHca . 
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71 



^■-7)-. 



Ocofptfifond Comptmon <j/ S>edk Sham 1972* 



Typt of occupaUoA 



li 

Is 




IS 



ii 
II 




hide 

and lei'hiika) 

Whnlcd (52] 7) 

5d^tffnploy«d manag«ri 
and odmtni^raion 

Sdbii«d mantis ai>d 
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SiMirctt! Cdkubtit) horn U.S Bumu of the CtiWii 136- 13tt). P^tntAfltf totals VAiy bum 100% dut to nxindin^ 
"I'uU'lirotr crmplufycd iiwc»m« eanittn, ch^o lat^or foKif. 



rom: Collins, R. The Credential Society , pp. 186, 187. 
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IS. 

Wn&t be Ac^unted tor by Diffcnnc^x in i.Q. 




Momi Til* Wc^tnid btt of och p«lr ihm (he mntiftJ 
pfl«^bUltjr thai L mfk It fit ibc top Mt^ «l thi iitokaie df^lrti 
buriOM if h« It la a Hivtn dMile Of wcieteonomU bac^voumJ. 
Tlli rilhi*1lii»4 thguv* tiM «»uit;aicd ProtoabMltr diat a mtf 
t» ttt dw t«i* AftH of lb* IrMOPM dtt:ribuiJ«n k( h« haa a<t»fi 
cKilAtOOd JQ anOli lit A diok «f td<iM»>cMink fruk- 

NWa ihif Uia of «nr 1^ w my «h<r»int 
ihii Utt in«uv/McJntdOfio4Me hackmund n1aiio«)fhip li al^ 
meit tbt i0iot tor JiHliTldHtti villi tdwiJctl IQ> 4a tat all 
MifWutli. 

MMNJl Non*Ncsio ffliltf «f oofHacm UcKpWfkd: 4fid 
35-44 f«an, 

•OIMe IsnsMt BovlH 1^ Valarft >felMR, **Tht tohcrliafKi 
of K}* ati4 thi [ntirftnaruiional TfanimiiitM «f E^^onomic 

N«i I. ftkrvarr m4. 



Fromj SowlflSi S. and H. Girttls. SchooHrtg in Capitalist America ^ 
p- 121. 
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16 , Relstlenshipt b§tween Chtracttristlcs cf Native White Nenfarm MatsS 

Aitd 25>€4 in 1 S52, Based on Observed Corralwttent 




R§Uitionshlps tetvt^ee/) Chtrwettrlstics of Nttlv§ White Nentam Males 
Attn 2S-€4 In 1962, Based on "Trut" CanelatSons 




POPED » Father's Education 

POPOC = Father's Occupation 

IQ » Early Cognitive Skills 

ED » Child's Educational Attainment 

AFQT = Child's Later Cognitive Skills 

OC = Child's Occupational Status 

INC ~ Income 



From: Jencks, C. et al. Inequality: A Reassessment of the Effects 
of Family and Schooling in fanerlca . P. iag. 
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17. HIGH SCHOOL OftOP-OUT RATE OF INDIAN STUDENTS IN SOUTH DAKOTA - 
19.75% (or four [4] times that of the non-Indian drop-out 
rate). 

NATIONAL DROP-OUT RATE FOR INDIAN STUDENTS The median for 
schooling coinpleted by Indians Is 9.8 years; only 1/3 finish 
high school. 

COLLEGE DROP-OUT RATE FOR INDIAN STUDENTS IN SOUTH DAKOTA — 
Of the students who did not enroll* or who dropped out at 
second semester* 69% are freshmen. 

GRADUATING SENIORS NATIONWIDE - 68X go on to college. 

GRADUATING INDIAN SENIORS NATIONWIDE - 35% go on to college. 

UNEMPLOYMENT RATE — Ranges from 40X to 70%, depending on the 
reservation. 

MEDIAN INCOME — $3,000 to $4,500 lower than the average non- 
Indian. 



From: Indian Education* Office of Curriculum and Instruction* 
Plerret S.D. , 1979. 



71 

76 



18. ACHIEVEMENT TEST DATA: Indians 

• Comparing all tests and grades, Indian students scored 
lower than thosg in tha non-Indian sample. The amount of 
difference) however, varied by test and grade level. 

• Achievement test data on the Indian students In the sample 
Indicated that the disparity between the students' expected 
performance and actual performance widens as the student 
advances through the grades. 

• The eleinentary level reading scores were generally higher 
than language arts and math scores. Conversely, math 
scores tended to be higher at the middle school and high 
school level. 

• Most of the achievement test data Indicated serious 
developmental delays for a significant number of Indian 
students. 

• National Curve Equivalent (NCE) scores were derived con- 
verting the Indian test data to a single comparable scale. 
Across all grade levels, test time periods, test levels, 
and test fonns. the average NCE for Indian students fell 
between S and 7 points below the NCE midpoint (equivalent 
to the 50th percentile). Non-Indian students were general- 
ly 20 points higher than Indian students. 

• In analysis of achievement test performance by income 
levels. EMS found that Indian students participating In 
the free lunch program performed below those Indian stu- 
dents who were not receiving free lunches. 



From; Minnesota State Department of Education. Indian and Bilin- 
gual Needs Assessment, 1979. 
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19, 



Tht DUrmnot Bctwm Stltctid RidtiyEtluik Gfoup 
Aehiminnit inil tht AeUtTtrntnt of All 17-YtftpOldf 



Piret nti|i poLnti Stindiid Error Nunbtr of 

Difftrtoe* From of tht Studtnti 

tht AcUirtmtat of Dlfftft dm 
AUl7-Yt«r-01di 







Social Srudln 






Whiti 
BiKk 
Hispule 


2.39 
•13.56 
•13.12 


Sdenee 


0.21 
Q.$6 
1.13 


22,690 
3,464 
1,259 


White 
BiKk 
Hlspinle 


2.13 
•10.32 
•11,08 


MethesHdei 


0.20 
0.61 
1.08 


20,370 
3,936 
1,105 


White 
BUek 
HUpule 


3;63 
•19.83 
' •14.36 




0.32 
0.60 
1.02 


25,427 
4,999 
1,376 




Ctrwf nd 


Oeenpedoaal Dfttlopntnt 




White 
BUek 
Hltpule 


2.19 
•15.96 
•7.65 


Retdlai 


0.19 
0.89 
2.08 


20,892 
3,087 
729 


White 
BUek 
Hiipule 


2.78 
•16.44 
•1 1.42 




0.22 
0.74 
1,54 


16,301 
3,523 
550 



Fi^m: Hispanic Student Achlevament In Five Learning Areas. 1971-76 . 
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20. 



The Diffmnca Betwftn Seltcnd Hlspink Group Achl«Yement and thf 
Achievement of All 17-Vftr-Oldl (In School) In Fivt Leunlng Afttt 



Perctnuge Potnts Standard Error Number of 

DlffHrtnca From of tha Students 

the Achievement of Dlffirence 
AU 17*Vea^0ldl 

Social Studlei 

AUHispanic JT-year-oJds -13.12 1.13 1^359 

Northeast -17.84 3.04 '»2S 

West .|2JS L22 963 

Male -12,03 1.48 587 

Female ,14.06 1,38 672 

Paients not graduates of high school *\ 5,47 1,24 604 

Parents graduates of high school -6.8 1 L42 403 

Science 

AUHispanic JT-yeai^olds -11.08 L08 1J05 

Northeast -13,67 2^02 170 

West .10.87 1,25 839 

Male •8.32 ' 1.47 501 

F^ale -13,40 LOS 604 

Parents not graduates of high school -12,36 1.30 536 

Parents graduates of high school -7,42 1.72 262 

Mathematlci 

All Hispanic 17-yeai^olds ^14.36 1,02 I 376 

Northeast 7.20 2! 1 3 214 

W«t ,|4J0 r22 1,033 

Male ,10,67 \M 634 

Female .| 7.94 0,75 742 

Parents not graduates of high school -1 5,52 1.56 668 

Parents graduates of high school •8,66 1,54 449 



Career and Occupational Development 

All Hispanic 17-yearolds 
Northeast 
West 
Male 
Female 

Parents not graduates of high school 
Parents graduates of high school 



All Hispanic 17-yeat-olds 

Nottheast 

West 

Male 

Female 

Parents not graduates of high school 
Parents graduates of high school 



From: Hispanic Student Achievement in Five Learning Areas. 1971^75 . 
p. 24. 
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-7.65 :.08 7:9 

8.76 98 

-6.84 1,86 SJ3 

■9,1 7 :,79 37; 

•3.67 Z.3S 357 

-10.:6 :,7I 356 

-2.80 MS 31 1 

Reading 

# 

■11.4: l.f4 sso 

■1 1.75 4.:6 103 

■11.06 ).79 409 

■13.39 1,70 :83 

•9-0 1.65 , :67 

■1186 1,38 :64 

•6,91 1,89 :38 



21. 



Tht Diffir«ne« Bitwtin Seltcttd Bl«ck Group Achiiviment ud tht 
Achitvimfnt of Ail 17*YwOldi In Fivt UtminB Ai lu 



Ptroiiitigt Pointi Stuidaid Error Number of 

Dtffirtnoi From of thi Studtnti 
tht A«hlivimtiit of - Difftnnet 
Ail 17*Yitr-Old4 

Social StudiH 

AU black l7*y«iMldt *i2*$6 0.36 3,464 

Nonh««it *I0.48 0J3 605 

Wait .13.1? 1.21 336 

MaJi *13.31 0.71 1,332 

ftmwlM .13.72 0.71 I,? 1 2 

fmnu not gr«diut«s of high lehoo] •] 6. 1 2 0.76 1 ,20 1 

Pitenti grtduttei of high ich^ol .9.93 0.33 1,889 

Seftne* 

Ail bliek )7*yeiNDldi •10.32 0.61 3,936 

NorthiAit •9.28 LOl 380 

Weit .10.93 1.13 378 

Maifl ^,9S , 0.73 1^738 

ftmwk .1 \A\ 0.72 2J78 

Pmntsnot |radutt« of highiehool " •lO.O? 0.36 ],281 

Pirentt grtduit« of hi^ school ^9.23 0.87 2,088 

All bliek 17^y««r-oldi .19.83 0.60 4,999 

Northeut -16.38 L79 774 

W«it -19.86 K03 713 

Mate -18.33 0.71 2^243 

F«m«le '20.73 0.72 2,736 

Pmnti not graduatn of high school -23.67 0.79 1,67) 

Pirentigrtduitfiiofhigh school -16.82 0.39 2,628 

Cartir ind Occu|»tionil Danlopitttnt 

AU black t7-yaa>ol4a -15.96 0.89 3,087 

Northiut -13,37 2J4 328 

Wait -17,71 1.39 466 

Mala -17.34 L:9 1,472 

FemiJa .14.83 L06 1,613 

Paranti not graduates of high school -18.00 L29 ],003 

Parents graduates of high school -13.67 Ml 1,814 

Reeding 

AJlbleck 17'yaa>olds -16.44 0.74 2,323 

Northeast -13.99 K61 331 

West .20.01 3.33 233 

Mele .19.24 0.90 1,170 

Female .14.03 0.^4 1,333 ■ 

Parents not graduates of high school .| 9 J9 0.83 794 

Parenu graduates of high school -12.98 1,03 1 420 



From: mspanic student Achievement In Five Learning Areas* 1971-75 . 
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22. 



Ltvtl of Southeastern Blick Achievement, Age 9, In Rtlilloit 
to Percitnt'White Composition of the Schoob, 1970 and 1973 



IfTO 



1973 




Level of Southeastern Blaek Achievement, A^ I3,ln Relation 
to Percent'Wilte Composition of the Schools. 1969 and 1972 




1 b Dtfflvmit hf«Mt*WhHe Gilif0clN 



Level of Southeastern Blaek Achievement, Age 17« In Relation 
to Percent'Whtte Composition of the Schools^ 1969 and 1973 



60» 



\m 





s 



From: Science Achlevament: Racial and Regional Trends. 1969-1973 . 
p. zo. ■ ■ 
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APPENDIX C 

NATIONAL ASSESSMENT PUBLICATIONS 1« 
READING, WRITING AND LITERATURE 



The following oompendlum is Intended to provide 
snapshots of naEP publications relating to Xlteraey, 
AH publications are available from HAEP offices in Den- 
ver, 



77 

82 



AsaeafliBent(s ); X969-70 Writing AssessTOnC 
' 1970*71 Shading Assessi&enC 
1970*71 Llceracure Assessmenc 



Aty Level(,a) 3 9, 13, 17 

Variables : Regloa. Sex. Race, Parental Education, Size and Type of Community 

Data: * Percent of Acceptable/Unacceptable Responses, Ten Vrlclng Tasks 

* Percent of "Yes'* Responses to Questions about Writing 

* Ifuakber >!echaQlcs Errors per 100 vords of Vrltlng 

* Percent of Respondeaca Correct, Literal Coop re hens Ion Sentences, 

Imperative Mode 

* Percent of Respondents Correct, Literal Comprehension Sentences, 

Decl?ra£lve 

* Percent of Kespoadents Correct, Inference Sentences 

* Percent of Respondents Correct, Recognizing Specific Literary Uorks 

aod Characters 

* Percent of Correct Responses to Objective Items, Understanding tmag- 

iuatlve Language 

* Percent of Correct Responses to MC Questions Requiring Vrltten 

Statement of Supporting Reasons 

* Percentages of Oral Responses Judged Adequate In Each of Four Hajor 

Response Categories 

* Percentages of Total and Adequate Vrltten Responses Classified 

According to Five Response Categories 

* Percentages of '*Yes" Responses to Questions About Llteraty Eteading 

Habits and Attitudes 

Other Content : Basic description of Hatlonai Assessment of Educatloaal Progress 
Htltlng, Reading and Literature Objectives 
Reading Passages used in assessnient 
Literary Works used la assessment 
Discussion and interpretation of results 
Discussion of assessment iDpilcatlons 
Suggestions for classroom teaching 

Related Material : All first assessment reports in Eteading, Vrltlng and Literature 

Co^ents: Several things about this book — commissioned by HA£P — make it 
valuable for the researcher la any of the language arts* First of 
all it presents a very handy succoary of the first assessment results 
in reading, writing and literature* Secondly, it presents a reanai^ 
ysis of the reading assessment results from a psycho! Inguls tic point 
of view* Third, the healthy skepticism about testing that pervades 
the book penolcs the reader to put the results into a broad per* 
spectlve* And fourth, the results and the f^ssessment strategies 
are applied to current classroom instructional needs by someone who 
knows those needs well* 

IfAEP Trading assessment results, presented by N'AEP in terms of 
median percentages of success and broken into 10 separate volumes, 
appeared to prove that Che va^t majority of people assessed were 
having no problems with reading, Mellon reanalyzed the results 
in an Ingenious way and came up with less sanguine conclusions. 
He assumed that each multiple choice question required the trans* 
^formatlonal co:^ining of ste:i and choices and then the truth test* 
lr,f> of Che four resultiog sentences. So he turned each question 
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ioto a ssntenct, cac^agorlzcd chtt senCei^ce in tenss of its aiode and 
its linguistic difficulty and rank ordered the sentences by per-* 
cencage of succetSt Such a procedurfl vas inCercsting in itself and 
ftlao r«vttalfid thtt skill In rsading higher levels ''Inferential** 
oatftritl was not tisarly ss wldaaprsad as had been supposed by 
psopls who had anUyzftd che resulCs asrlier. Mallon*s model of the 
thought procasssfl involved is answarinS multiple choice "comprehension** 
itaaa rulsas sarious naasursmanC quastlons for anyone involved in the 
taatiDft of raadins. 

Hallon*^ discussion of chs writing assessment is useful to the teacher 
or cha tasting tyro, but tha rasaarcher will find lictle there of 
nota* Uia criticisn of various aspscts oz the assessment ara famll-* 
lart it was atoalsad; iti forced rolc-*playlng probably kept results 
dacaptlvaly low; sxtrspolations frooi the mechanics study ara Jikely 
to ba blown out of proportion and alsunderstood; and ^IA£F needs more 
rhetorlcAlly-orisnted tssks. 

Tha discussion of tha litaraturs assessment clearly explicates the 
general difficulties involved in such an enterprise and the specific 
ones encountered by MAEP. Mollon feels that the attempt to measure 
response to liceretura consunsd more Clme than ics vague results 
Justify^ Conssquently, rsseerchars in this popular area should attend 
to thle critique* 

Mallon*s sumary exaoinas such problem areas as motivational lev^al 
of studSDts being essessedi comprehensiveness of the assessments* 
utility of NA£P data end application of :jA£F materials at state or 
local levels. 

All in all, this book constitutes a fine* level^^headed explanation, 
end critique of ^ia£P work in the lonsuage arts. Irs greatest 
lialtstion is chat it was urittsn prior to the reassessments of 
vrltlnsi rsadins and litareturs. 
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TlEli i Reading: Siinoiarv Pata i National AitesaQQnt ct tducatlcnal Progress 
RaporE 02rK-00, ERIC ^ ED 094 359, July, 1974, 5a pages. 

Typo of Roport i Ttchnlcal suanoary 

Aaaaftiaantffl) i 1970-71 Reading AssesBfiflDC 

Aftt Hvel(i) i 9, 13, 17, Adult (26-35) 

7arlabl*i i Raglon* Sex, Kace, Parental Educttloni S128 and Type of Community, 
Sl2« of CoDmninlEy 

lt«i Typtf{i) i No Items 

DatA i H«dl&ii varldble group (malos, blacks, eEc.)^ difference (in percentags 
poinEa) froA national level of parfonaanca for aach age on each Reading 
chaoM and aach Reading objectlva 

Othar ContcnC ; DaacrlpElon of Reading objaecivaa. 

DaacrlpElon of Reading axarclaaft. 
Daacrlptlon of Reading chamea. 

Sourco Material Available eo Researchara i Raw raaulEa on 1600 BPI, 9-track 
4aCa capaa uaable only on IBM 360 or 370 hardware with OS or VS operating 
ayaEftflia 9 $94.00/tape. 

Related Mattrlal i OZ-R-Ol* UndersEandlng Iforda and tford Relationships . 



02-i^02, Graphic MAEarlala . 
02-R-03, Written Plractiona , 
02-R-04, Reference Matarlala . 

02-R-05, Gleaning Slanifltant Facta From Passages . 
02-R-06, Main Ideas and Qrganl gat lou t 
02-R-07, Praxjing Infarancaa . 
02-R-08, CrlElcal Reading , 
02-R-09, Reading Rate and CoffiPrehanSlon . 
02-R-30* Recipes^ Vra&gftTlj- Reasoning and Rat^e . 
02-GIY, General Infornation Yearbook for Reading and Literature. 
Mellon, John* Satlonal_AAAAiaTnaBt-"and the Teaching jf 
English . NCTE, Urbane* Ullnoia* 1974. 



Comment a t Thla rnport consists of 40 tables which dlaplay median percenEaga 



differences from naEiooal performance and also display "directional 
tendencies" of each variable group. These laEEer reveal at a 
glance t/lilch, on a given thome (a.g. » UndersEandlng ^.aln Ideea» 
aEc.)ft a group (females, lnner*clty clwellera» etc.) tended to per* 
fouv above, below or at Eha national lavel at each age. 

The emphasis of the report is entirely upon group differences; thla 
fact defines its uElllty for people Interested In a highly absEract 
look at overall results and lES irrelevance for p<?oplG looking 
for exercises or exercise- level resulES. 
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Tlcl< t National Assesstnenc olf £ducacional Progress Report Bi Wrlclng Mtchanlcs . 
ERIC & ED 062 323. 19;2, 202 pagia. 

Typg of ^flPorc : Selected ReSuLcst Pri&arlly for English Tifich«r« 

AigtaiaanE(^) : 1969*70 Vrlclng AAiflSioenc 

Agj Lavalfa^ ! 9, n, 17, AdulC (26-35) 

V»r iablia t ffacional Data Only 

lEaa Tvpa^s) i Essay and LeCCer 

Dac> ; Nusbar of different vordi In high*, mediunh- and Iaw«qualicy aa&ayi« 
Total nuc:ber of words in high*, jcadium- and Icv-qualicy etiaya* 
Kaan word length in laccara. 
Standard deviacion of word langCh in leccers. 
Total ntiober of sencencea, 
Nunbar of declarative aenCancat. 
Nuttbar of incerrogaclva sanCincas. 
Nunbar of imperative aancancasi 
Haan tentance lecgch in words. 
Standard deviation of sentanca langch in vords. 
Nusbar of paragraphs, 
Maart paragraph length In santcncaai 
Srandard deviation of para^rapU length in sentences* 
Rilacive pronouns* 
Uaad position gemnds* 
Head position adverbs. 
Head position past participIaSi ' 
ifuobar of colons, 
Nuttbar of semicolons, 
Nuttbar of parentheses. 
Nuabar of quotation marka* 

Obvious misspellings of common vorda (a common word ift any word appearing 

at least once every Cvo papara). 
Obvious misspellings of proper nouns. 

Htatpalllngs of uncommon vordt (uncocmoon vords appeared lata Chan once 

avaty tvo papers). 
Error counts in paragraphing, puncCuation> capitalization, santenca 

ttructtire* agreement, and tpalling and word usage. 

Other Content : Essay assignmants used for mechanics study* 

Explanation of all qualitative and quantitative analysea 
performed* 

Characterizations of easaya at different quality Isvala by 

panel of ''experts/' 
Sample essays. 

Relationship 0^ study aampla to full populatlona of 9^yaar- 
oldSp 13-year-olcls, 1 7«'year-*olds and adults. 

Source Hatarial Available to KasaaEchars r Contact NAEP 

Related Matariiils : Hellon> John. S'ational AsseRsfient and the Teaching of 
English . L'rbana, Illinois, 1973, 
Slotnick, H.S., "Toward a Theory of Ccmputer Essay Grading/' 
Journal of Educational Measurement , Winter 1972. 

Comnants i From the entire distribution of about 2,500 essays, which had been 

holistically marked, XA£P selected s;;aiple papers near the 15th, 30th 
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«nd 60ch c«nclls3 for spsclal analysis. &ich l^aptr vas axAtolnid by 
tvo English csachars for vschanlcal amri an<l choas ware aggragacad 
laco *'arror counts. *' Than^ a panal of Judgas raad aach aac of papara 
and characCarlzadf iAprasslonlaClcally* cha aaca, p'tcclng cha arror 
counca Inco a mora aaanlngful partpacclva. Tha Judges daacrlbad good 
aa veil aa bad quallclas In cha i^aptra and dlscuasad vhac cha vrlcara 
vara cr:^lng co <lo and hov vail chay vara doing Ic. Flnallyt cha 
aaaays vara acannad by a compucar prograonad co provlda Infonucloa 
abouc fluancyt apalllng, dlcclori» aaacanca scnjccura^ punccuaclon 
aftd paragraph davalopaanc. Tha raporc pratanca raaulca of chla aCudy 
for an aaaty ac agaa 9r 13 and 17^ aa vail as a laccar vrlccan by 
adulcst 

Alchough cha raporc concalu a good daal of locaraaclng daca snd cofl^ 
mncary, much of ic can be auffiotd up In cha obaarvaclona chac good 
papara ara aaslar co raad* longer and laas arror rlddan chan poor 
papara, and chac poor papara sound &ore llica spsach chaa vrlclng. 

Thesa ara noc aurprlalng ravalaclons. Furcharttora* cha arrot cotuics» 
Judgas' Judgmancs and coopucar analysaa all supporc cha ganaral 
ordering of che papara tscabllahad In a hollsclc acoriag^ aa va would 
axpecc, 

Tha prloary vlrcue of chla raporc la chac Ic copblnaa quallcaclve 

vlch quanclcaclva approachas In an afforc co flsah ouc our undarscandlng 

of vrlccaa produces. Ics primary llablllcy conalacs In cha face 

chac Ic la a product scudy and Ic focuses anclraly on aachanlcs . 

This la axaccly whac Ic was auppoted co be* of couraat buc ctiara Is 

Juac so much one cao proflcably laarti wlchln such a llalcad parapecclva. 
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Tltl ft: Critical Tteadlng T SaClonal AssessmenC of HducaClonal Progress Reading 
lUporC 02'R-08, ERIC ED 073 387, May 1973, 169 paS^s- 

type ofURtPort ? Selected rtsulcs 

Aase*ga*nc<[g) t 1970-71 Reading ««9essaeaC 

Age Uv^l(a) i 9, 13, 17, Adult (26-33) 

VarlJiblftg i RegloQ, Sex* Rac«, Fflrflncal EducaCloa, Size and Type of CofEmunlcy , 
Size ox Commualcy 

ICmp TyptfB) ! Multiple choice; rtcognialng llterAty devlcf»» Inferring 

from metaphorip IdcnClfylng mood^ recognizing sCnicCure* drawing 
gpproprlaCe inferanciv. 

Pace validity* 

Batat PtrciSCtges of success on rtltased items for tach tgv and for variablt 
groups within age, 
Stmury disCribucions across til items, displaying msdian differenca 

from astional perfonuocs for each variable group at each ag^, 
Parfonaace comparisons on itams conmon to evo or mora age levels- 

Other jCootant : Introduction to reading assessment 
Items used in the asssssment« 
Discussion of salacttd items and thalt rasults. 

Sourcg ^tarlal Available to Resaarchtrs : Raw results on 1600 BPI, 9-track 

data tapas usable only on IBM 360 or 370 hardware with OS or VS operating 
systoos € $94«00/tape. 

RalatggL-Ha.taria^ : Reading add Litarature General Information tearbook ^ Oi-GFjf. 

1970 Reading Qbjactlves , especially Objective II, "Analyat 

What Is Raad,'' and XV, "Reason Logically From What Ts Head, 
Reci^es^ Vrappars. Reasoning and Rata . OI-R'SO* 
Reading: Smaaarv Data ^ 02-R-OO, 

HelXoDp John I N'ational Assessment and the Teaching of 
English , NCTE, tirbana. Illinois, 1974, 

Cdrmnents i Thare is some overlap here with the Literature asssssment, sines 
many of these items uss posms as scimuXi. Since critical reading 
involves going back to ths text after having raad it once and 
aoalyring it for graater understanding, and since, in order to 
oeaeure critical reading skills we have to fprce people to go back 
to the text and tsst four hypotheses — ws can never be sure 
whether they can do it on their own or not. This qualification (a 
consequence of the multiple-choice format) aeidSp the material here 
le worth examining because it demonstrates problems vith higher* 
Isvsl inference and it provides daca about adult readers* 
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Ticl^ l Functional Littra^! Basic Reading Farforaance ^ National MsessmCnC and 
RighC Co Read, 53 pt^es. 

Type of RePOtt ; Joint rcperC with Right to Haad 

Asst^aoantfa) ! Special Right to Head aisaasicents in 1974i 1973; Reading 
asacaaaantf 1970-71. 

Age Lavelfa); 17 

Varitblat i Region, Race, Sex* Parental Edutation. Sise and Type of Comtmnity 

Icca Type fa) : Hulcipla choice* lou-diff itulty items selected from the 1970*71 
item pool by Right to Kaad ataf f as "ftmctiooal literacy^ 
itens- 

pata i Haas percentagea of lutcess oa 64 itau for all variable groups, 1971 1 
1974 and 1975. 

ttoan percentagaa of luccesa on 64 iteout compared to deaired resuXtSi 
all three yeart. 

ttean percentagea of tuccess on 64 items compared to highest expected 

results, all three years, 
Kken percentages of sue cess on 64 items compared to minimal, standardiT 

all three years^ 

Haan percentages of success qq 86 items for all variable groups, 1974 
and 1975. 

ttean percentages of sutc^ss on 36 items compared to desired results, 
two years. 

Mean percentages of futtess on 36 items compared to highest-^escpectad 

results, tuo years* 
HesQ percentages of sutcess on 36 items tomi^ared to minimal standard!, 

tm years. 

Percentages of euccess for uni<iue exertises on vhith performance 

detlioed markedly. 
Percentages of success for uni<iu8 exertises on vhich performance 

Improved markedly. 

Other Content : Brief summary of results. 

Explanation of study. 

Source Material Available to Researchers ! All items used ate available upon 
request- 

Related ^t^tlal ; iVJt Stati^tlcal/Pocumsntarv Report — Sumarv Volume . 

Handbook of the Mini'^Assossgenc of Functional Literacy . 

ComaoQtS ! Right to Rti^d aotetted exercisss from the HAEP pool which they 

thought 100% oC Amsrica^s l7^ysar-olds should answer correctly and 
funded two "mini^as^essments of functional literacy^* (}lAFt} using 
those Items. This report presents the results of that stud^ and 
also makes the only N*A£P foray into **crlterion testing.*' The 
Right to Read exercises vere given to IQQ students proven to be 
Superior readsriT Their petcentags of suctcss on any exetcise 
wee taken to be the ^'highest expetted level of performance'* and 
the tesulta were recalculated using this index instesd of 100% 
«« one ctltorlon against vhlcK to measure literacy. Right to 
Read also d^tided that 757 vaa the minimum percentage of success 
to <iuallfy as funttlonally literataf and tesults for the variable 
groups were plottad against chis staofjard as wall. 
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Tillc i Lltarsturc AssflssotnEi Suaaflrv D^ita i T^aclonal Assessment of £ducdcionaI 
Progross Hiport OS^L-OQp £RXC ^ ED 079 635, June, 1973, 105 pages. 



Typi of p ; eporc ^ TechtileAl 4uimn«ry for sooevhat technically oriented audience 
Aaaaianatit fa> ! 1970-71 LittiraCure ASflessment 
Agt Uvtlfg) ; 9, 13, 17p Wult (26-35) 

VtriablK i RfgioB, SftK, lUcii Fftrcntsl Education, Si^e and Type o£ Community, 
Sizo of Cotaauaicy 

Ittn TvBifa) ; ^'o itios included in this report. It is based upon all the 
itft&s uAid in chs litflrature assessment. 

Data : Sumry diatributiona across all items in each literature report, 
displaying tnediaa difference from national performance for eadi 
Ttriable group at each age. 

Hidlan differences fron fkational perforoance for variable groups by 
Literature objectivaa. 

Performance compariaona of age groxips on comtnon sets of items. 

Other Content: Introductioa to Literature assessment 

Source MeCarlai Available to Researchers : Raw results <7n 1600 BPI, 9-track 

data tapea usable only on IBM i60 or 370 hardware with OS or V5 operating 
eyate&s 9 $94.00/tape. 

Related MeteriaH 02*iCIY, RiadinS_and_Llterat:itre General Xnfgraation Yearbook , 



Purvta, A., and Reppere, V,, Elements ot Writing About a 
Literary tforki A Study of Response to Literature , ^CTE 
Reaeerch Report 9* Urbana, Illinois. ~ 
Halloa, JohHp National ^Vsaessment and the Teaching of 

Enaliah . XCTEp Urbana, Illinois, 1975, 
Grifkdstaf f p Faye Louise, "The Xarional Assessment of 
Literature; A Review," Research in the Teaching of 
English , 9)li80-97, Spring, 1975. 



Comenta i This report collects the sujnmary information included in each 



Literature report end puta it all in one place. In addirion. it 
presents results by objective and presents some interesting age and 
variable^group conparisons across the range of Literature items. 
It makes for very dull reeding, but it does provide a useful over- 
view of results for people who are utwllling or unable to read 
all the Lilterature reports. 
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liiii^ Jlfavtng tnferoncas . KACional AasassmeaC of EducaClDnal Progress Report 
Typ< of ftaocarc! 3el«ccad rasulta 

Agj L«vgWi^! 13, 17t AdulC (26*35) 

Viriablcj; Ragion, 3e:ct Raca, Parancal Educacion, Siae and Type of CoisiBunity, 
Siaa of CODQBunicy 

Itan Typaftt)! Hulcipla choica; low* and high*laval infaranca <;u«sCionsr 
noQsansa word icamai 
Faca validicy* 

Dajas Parcancagaa of auccass on ralaaaad leasts lor aach aga and for variabla 
groups wichin aga« 
Suwoary distributions across all itaos* displaying madian difference 

from national parformanca for aach variabla group at each age. 
Parfomanca coaparisons oo itaiu coizaoo to two or mora age levels, 

Qthar Contanti Introduction to tha Raading asaaasaant. 
Itama usad in tha asaasaDant. 
DiacuasiOD of salactad itaffla and their raaulta 

_gourca Material Availabla to ftiinaflr^^^r«a Raw results on 1600 BPl, 9^track 

data tapaa uaabla only on I3M 360 or 370 hardwara with OS or VS operating 
systaos € 5$4,00/tapat 

J^^ftfd frt^t^r^ftji Raading and Litaratur a Ganaral rnfomacipn Yearbook . 02-GXY. 

1970 Reading Obiactivaa- aspacially Objective lit "Analyze ' 
Is Raad»" and IV, "Reason Logically From What Is Read 
itacipaa. tfrag^cra. R^aao nina and Rata > 02^R-30. 
Reading: Stieinary Data . 02-R*00. 

Mallon^ Joho, iiational Asaasanant and the Taaching of 
English , NCXe, Urbana* Illinois, 1974. 

talti Parcantages of auccasa ara a good daal lowar in this report than 
thay ara in tha othars, indicating that reading achievement must 
ba lookad at in as nany wa>2 as possible in ordar to draw legitimate 
conclusions* Claarly, most Amaricarts can raad auch low^leval 
infaranca materials as ara nacasaary to "oaka do"* or **survive" or 
graduata**' But ability to comprahand evan BOdastly complicated 
taxt ia not «o widaspraad at any aga and is appallingly skimpy 
among cartain variable groupa. This fact^ overlooked because of 
the relatively good perfonaanca on the entire assessment, 
bears further atuJy. Like the rest of tha Reading reports, this 
one« too* ia uaeful es a rare source of data oo adult reading 
ability* 
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Title ; Reading In Aaerlca: _Jk. Perspective on Two Assessments , National 

Assessoent q£ Euacatlonal Progress Report 06*^^01, HRIC £r ED 128 785* 
October, 1976, 30 pages, 

type of Report ; Interpretive overview for geDeral audience 

Aaseasnentfs) : 1970-71, 1974-^75 Reading aSsessaeDts 

Age Uvel(s) : 9» 13* 17 

Variables : Region, Sex, Race, Parental Edt;catlon (post-high- school and no- 
hic^-school education only). Size and Type of Conununity (high 
metro* low metro* mral only) 

Iteft TypeCs) ; A few items are shown as examples, but most of the Items used 
to measure change in reading performance were not released* 
The majority are raultlplcs^cholce questions* such as those 
displayed in reports about the first assessment. 

Data: National and variable group mean percentages of success, 1970-^71 
assessment* 197A"*75 assessment, for literal comprehension items* 
inferential comprehension and reference skills items* ages 9, 13 and 17* 
National percentages of success on selected exemplaty item?. 

Other Content : Interpretive comments about changes (and nonchanges) in 
reading performance becveen the two assessments by Roger 
Farr, Hllllam Blanton* Carita Chapman* Diane Lapp, Hayne 
Otto an4 Robert Tlemey, 

Source Material Available to Researchers ; Some* Contact Hayne ^^artln at KAEP 

Related Materials : 02*R*01, Understanding ITords and tford. KelationshlPs # 

02-il-02, Graphic Materials - 

02-K-03* Written Directions , 

02-R-OA* Reference ?late rials . 

02-R-05, Cleaning Significant Facts From Passages , 

02-R-06> Main Ideas and Organization , 

02-R-07* Drawing Inferences . 

02-R-08, Critical Reading , 

02-R-09* Reading Hate and Coop rehens ion > 

06-R*00, A Succtary ot Chanaes in Reading Achievement , 

Comment s; The second Reading assessment revealed a significant Improvement 
in reading ability at age 9 but little change at ages 13 or 17, 
At all ages there seems to be a problem comprehending basic* 
literal written material; but comprehension drops off quickly as 
soon as the tasks become difficult. These facts form the focus 
of a conversation about the assessments and their meaning among six 
distinguished reading experts who have examined the materials 
upon which the results rest. Researchers will find hints about 
potential directions to- pursue* but they will not find sufficient 
data in this report to satisfy their curiosity. because it vas vrltten 
for a general audience more interested In eicperts' opinions than 
in statistics. The much more tomprehenslve summary* 06^R*00, would 
be a better place for researchers to start. 
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li tie : Reaoondintg to Ljtcrqture t fJationdl .^sessment of Educational Progress 
Report 02'L-02, EHIC 4 £D 077 020» 1973* 231 pp* 

Type of 3teport : Selected ?t5»ult3 for General Audience 

Ajjses spent (si : 1970-^71 Literature Assessttent 

Age Level(s) : 9» 13» 17, Adult (26-35) 

Variables: Region* Sex» Size and Type of Connnuniry, Race* Parental Education, 



Item Tvpe(s) : Oral Etesponses to Literary Works* 

Written Response to Literary Works* 

Data ; Percentages of people responding to literary works in the following oode^: 
Engagement Involvement 
Formalisclc 
Interpretive 
Evaluative 
Retelling 

Percentages of responses Judged '^inadequate," "barely adequate,^ 
"adequate'* and "superior*" 

Other Content ; Literary Works Used to EUclt Responses* 
Explanation of Response Categories* 
Sample Responses* 

Source Material Available to Researchers : Haw results on 1600 BPI» 9-ttack data 
tapes usable only on IBM 3^0 or 370 hardware with OS or V$ operating systec^ 
@ $94,00/tape. 

Related Material: 02-GIY» Reading and Literature General Information Yearbook * 



Purves, A*» and Reppere* V*» Elements of writing About a 

Literary Work: A Study of Response to Literature , :;CTE 

Research Report 9, Urbana* Illinois* 
Kellon, John* ^iational "Assessment and the Teaching of 

Et^ftlish , SCTE, Urbana, Illinois* 1975* 
Grindstaff, Faye Louise* "The ^;ational Assessment of 

Literature: A Revie?*," Research in the Teaching oC 

English , 9:1:30-97* Spring* 1975* 



Copacents ; This study Is based uPon the response to literature system developed 



by Purves and Reppere* That system* too unwieldy to employ with a 
large sample of papers, was modified from a sentence-^bysentence 
analysis to a holistic analysis of the entire response* That is, 
scorers indicated that the primary thrust of the response was either 
evaluative or interpretive or whatever* In addition, scorers ranked 
each written response on a A-^point scale from inadequate to superior* 

tiine* year-olds were read a story and asked a series of questions about 
it* Their tape-recorded responses to the questions were transcribed 
and categorized* but the results are rather oe^sy. The interview 
questions tended to force particular response modes and the attempts 
to evaluate responses were complicated by the brevity^ of so cany 
answers* All in all* the 9-year-old data is interesting but incon-^ 
elusive* Thirteen-year-olds, 17-year-olds and adults were given 
poems and short stories to respond to In vritin^, and these results 
are meatier* 



Size of Community 
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This report raises Icportant questions about the relationship between 
research and assessoient* Researthers can design studies like this 
one much inore coaprehensiveiy than NAEP can* They can give the stu- 
dents core tloe* find out oore about them> control for various factors 
and give subjects several opportunities to display their repertoire 
ot response tnodes* :iA£? can give students one 15-minuce shot at 
responding within a test-like 'situation* 
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Title : Explanatory and Persuasive Letter Wrltioa , SAEP Report 
EXIC ^ED 135 006, February 15^77 



Type of Report : Selected Results for General Audience 
Asgegament Cs) : 1969-70, 1972-74 Writing A3desaine??.t 
Afie Level (s) : 9. l3t 17. Adult (26-35) 

Varlab les : Males* Fenalest BlackSt Whites, Low Metro, High Metro 

Item Typefs) : Opeit-ended: personal letters (audience specified); fill In the 

blanks — greetings closing, addressing conventions; letters to organisa*- 
tlons (situation specified); employipent application letters. 

Data: National and Croup Varlabl^: percentages of 7ntmg5ter5 writing "inadequate , 
^taaic,'* "coiopeteot" and "eaccellent'* letters; percentages of youngsteis 
deMQStrating control of various rretotical skills (primary trait scores): 
cban^ej In some percentages betweer 1969 and 1974 assessaents. 

Other Content : Examples of exercises and responses 

Source Material Available to Researchers ; Contact ElAEP for access to original 
essays or reproductions. 

Belated Material : Writing^ 1969*1974: Technical Reporr for complete documen^a-^ 
tlon of results including standard errors of percentages. 
HAEP Writing Objectives . 1969 version and 1974 version, ' 

Comment.s : Soiae Items are Interesting In themselves, but there is no summary 
ixiformatlon across writing assignzreats , making generalisations about 
overall writing competence impossible. Ho two exercises were scored 
vlth exactly the same criteria* AJult results appear for only rwo 
IteiBft* These results also appear in Adult Work Skills and Knowledge 
HAEP Report 05-COD-Ol, 
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TlcU i CqnTal Infonnaclon YgJ^^^ook , SA£P RfipotC 02-GlY, £R1C ilO C72 iU, 
May 1972. 

Type of R<pott : TicKalcal Bickftouod. 
Aisciioenc(s )t EtA«41ng> Llc«r«curfi) 1970*71. 
Am Uvl fs)i Applies co 9, 13, 17, Adulc <26-35) . 
VttlablftS i I>«ii:rlbcf all virlabl«6 usid in 1970*71. 
Itfffl T/p<y g) ; No ic«u It^eluddd. 
Pau l Hon*. 

Conctnca i Background infonucion about chc 1970*71 Rftadlng and Llccracurc 
aaaesHftCM includliig: 

^ dcvaloptMnc oi ot icivta 

- dcvclopwnc of cxf^reisic 

*. dtflaicloa oi variable groupa 

^ laspllng proeadures 

^ adolAiscrcclOQ procedures 

- ecorlAg oi icen 
* decs proeaaslQg 

- data analysis proeeduree 

- ssi.ee cion ol exercises lor reports 

- cavescs about sbssssdsqc dscs 

- vlrcuss acd liabilities of dacs sdjuscosncs 

tolstsd htacerisl i All Reading and Liceracure reports. 

Coflmancs i Ksessaary background for any ssrious rssasreh uaing chs dacs 
fron thess assessasocs. 
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Titl e; A Survey of Rlgdina Habits . Report O2-l*-04» ERIC tfED 073 366. 
May 1973 



Type of Report : S«l«ct«4 Etasults for General Audience 
Maesament (g> ; l970»7l Lltereturc Asaessnent 
A ge L^veH;s ); 9» 13» 17. Adult (26-35) 

yarlablea : Keglon, Sex* &lce» Parental Education, Size end Type o£ Community 

Item TypeCs) : Survey (group and individual); Yee/oo; egrea/dlsagree; wrlcteti 
and tpokto responses to accltude mad cxptrlance questlcne 

Da£a ; National perceouges reporting various attltudea toverd and levels of 
InvDlv^fEieot with lltereture, as well as vsrlous kinds o£ literature 
most of:;en resd* 

Variable group reaulte expressed both as ebsoXute percentages and as 
differences from the national percentages on «ech Question* 

Variable group resulta expressed as oedlsn differences froo the natloa 
across all questions. 

Sample spoken and vrltten responses to survey questions* 

Standard errors £ot ell percentages Iq an appendix- 

Other Contents : Survey Itemst Attitudes toward lltersture 

Reading Inventories (novels, poems^ Plays* 
non*flctlon> short stories, criticisms 
hlograPhy* magazines) 

Involvement with lltersture (ce-readlng* 
reading another book by the author of a book 
you enjoyedf reading a book and seeking to see 
a oovlet play ov television version) 

Should movies be tsught In English classes? 
Why or why not? 

Source Material Available to Researchers ! Raw results On 1600 BFI* 9-track data 
taPes usable only on IBM 360 or 370 hardware with OS or V5 operating 
systems ^ $94.00 per tape* 

Related Material ; 02*L-20, LI terature ; ^uTrnnjtYv Pf t;f * 

02*CIV, General Tnfarnation Vearbook . 1972. 
Johnson, Slsor.* '*Hov Students Feel About Literature*"* 
Aioerlcon Education * Vol* io* 3* April l97i* 
Kfillon, Jolm> National Assessaent and the Teaching of 
English, NCTE* Urbana^ Illinois, 1975* 

Coiaaents ; Because there Is little specific laforiMtlOn available about national 
preferences In reading (by genre and by level of sophistication 
within genre), this Is a unique and very useful rt^port. Sample 
responses to Questions like "Do you think uovles should be studied 
as Part of English classes?" make good reading. 
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tl , tly t ReclPaa* Wrappers, RatsORlnc and Rflt «* NAEP RtadlRg Report 02-R-30t 
ERIC 092 April 197^ 



Tyo« of Report t DlgtsC of RfiadlRg AssesamcnC Resales for General Audlenc^f 
iRcIudlRg loCerpreClvt Chapter 

AstfSf n*tat(s) ; I970»?I ReadlRg Assessm^RC 

Variables ; Re^lOR* Seac* Race* Parental EduCaClon* Size aRd Type of CommunlCy, 
Special acctRClon davocad Co raca aRd sex diffarCRces la parformance 

Itan Tvpe(s) « Hulciple-cholea ICeos appaar aa examples 

Daca t No resulca daCa chaC does noC appear 1r oCher reading reports • 

Other CftRtent i IncarpreClve dlacusslon of results. Table dlsplaylRg number of 
e^cerclaaa In each reading report ac aach age level, MedlaR dlfferencea 
becveen group aRd Rational result! for all iceos la ReadlRg aasessnenC. 
Percencagee of blacks and whlce^ In each pareRCal educaClOR eaCegoz?* 

Kale ted ^tacerlal i 02*GrTt RaadlRg anH Lilaratura CeReral iRfortnaeiOR Yearbook , 
All lading raporCd, 
toadlng Qblecclveg - 

Mellon* JohR* KaClonal AaaessBteitC and the TftaChiRfl of 
English t NCTE, Urban** Illinois* i975, 

ComaRts : Thia voluoe revl«vs results published la Raports 02-^R*01 through O2-R-09 
wlch particular emphaale upon dlffareRtlal parfomaRce of ualas ^od feaelee 
and bleckJ *nd uhltesi It concludaa wlch an iRterpretlve discussion of 
results. eooRg ^tary AnR Balrd* CoIIr Dunkeld, Olive Nlles, George 
FhllllpSt Harold Herber* David YerrlRgton and author DoRald Gallo, A 
useful overview of the resoles* but Rot sufflcleRt* Must b* complemeRted 
by the HellOR book* 
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Title: First Assesspent Literature Qbi^ctives, 1965, ERIC ff ED OAl 009» 21 pp. 



Type of ^e^qrt i Objectives Booklet 

Assessmen cCs) : 1970-71 Literature Assessmeat 

Age Levelts) : 9, 13, 17, Adult (26-35) 

Contents : Introduction to 7IAEP- 

Description of objectives development procedures, 
Literature Objectives: 

Ir Read literature of excellence 

A, Be acquainted with a wide variety of literary works 
5« Uuderstand the basic aetaphors and theaes throufS wKlch 
man has expressed his values and tensions in Western 
culture 

Ilr Become engaged in, find oeanlng in ard evaluate a Work of 
literature 

A* Kespoud to a work of literattire 
5« Find meaning in a work of literature 
C« Evaluate a work of literature 
III* Develop a continuing Interest and participation in literature 
and the literary experie^ice 

A, Be intellectually oriented toward literature 

B, Be affoctively oriented to literature 

C, Be independently active and curious about literature 
Kelate literary experience to one^s life 

Discussion of each objective and stubobjective. 
Description of relevance of each subobjective to each age group. 
Names of people associated with the development and review of 
literature objectives. 

Comments : These objectives* developed by Educational Testing Service in che 

oid-sixtiest represent a somewhat elitistt "belletristic" concept of 
literature, Tney assume that "literature" is a body of works rather 
than a way of using language and that "literature of excellence" can 
be specified in objective terms. Both assumptions are erne only in 
limited ways and both would steer assessment activities in a direction 
familiar to EXS test takers, Unfortucintely^ a nationwide assessment 
cannot assume that participants share knowledge of specific works and 
criteria for adjudging excellence; some of the objectives were bound 
to go to vaste, and some of them were bound to waste precious tlae as 
assessors tried to determiue how many Americans recognise Faul Bunyon 
or Job or whomever. 

The second objectivev however* was appropriatev though measurement of 
it vas no trivial or inexpensive task. 

On the whole* these objectives are interesting as a culminating expres- 
sion of viewpoint nurtured by university interestst propagated by 
college testing and sustained by the relative econoaic health of 
education In the 30s, £t vas a narrow vieu^polnt taat was already in 
question and endangered when the objectives were developed. By the 
time the assessment vas administeredv this perspective* by itself » vas 
obsolete. 
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TUU: Tteadtng Rac* and Conarehension. NA£P Rtfflidlng Rttporc 02-R'09t 
EAIC ^ £0 0/6 Decdsber -L^j2. 22S pp. 

Type of Rtporc r Stitcced Hesulcs for G^ntrtl Audleiaca. 

Aajf a«CMPC(< ): 1970*71 fading Assess&cac 

Age Uvil(a ): 9, 13. 17. AduXc (26-33) 

Variables ! Rtglotti Sexi Race ■ Parencal CducaciOQ* Siaa and Type of CcMi^sr^ityi 
Facher'a tducacloa* ^^oche^'s edufiaclOBi Rtading aacerials in the houJt, 

Iron_Typ*ja ) i RaadlaS Race Measures} mulciple^choict coapreheosloci quesciooa 
locualog recall o£ decailai 

Daca i Ptrctttcagas of people ac each aga lenl and in each variable group 

reading favar chan lOO * 100*199 w.p.m^t 200*299 w^p.Q. and more 

than 300 vipio^ 

Diacribucioas o£ reading .jes fron 0 co over 73 v^p.m* across differaa^ 
. paaaagaa ac each ^Se level, 

Spachti LOfgfli Fogi Dale Chall and SMOG Raadabilicy Indices for passages. 

Percancages of people at each aga laval and in each variable group cor* 
raccly ansvaring comprehension quascionSf^ crosaad vich reading raca (<»g> 
732 ol tha male 17-year'Olds who r«ad ac 100*199 v.p.s^ correccly ansvarad 
four or fiva ouc of five compr^ehanaion ^tiaacloga alcer reading a passage). 

Analyaia af exceptionally £asc (> 730 v.p.mO and txcepciooally. slow 
( C30 tf.p.a.) readers, 

Parcancagaa of people ac each aga laval and in each variable group: high 
cnfflprehenaion within reading racoi 

Pttrcancagaa ol people at each age level and in each variable group: race 
vichia comprehension. 

l7*yaar*old an4 adulc performance compared on conoa icems. 

Diacuaaion ol items and resuXrs* 

Ochac Concanc : Pftaaafi^s used for assessnanc o£ race 

Icama used for assessmenc of coaprahttnaion 

Relaced Hacerial l Ol-^CIY* General Informajcion Yearbook s 1972, 

02-K-30, Rec£pcs> Wrapp^ra. Reaaon;Lng and Race : A Dlgcsc 
o f Che Flrsc ReadinSi__Aaa»ssffienc » 1974. 
?2^R'Q0^ Suinm.iry Daca < 1974^ 

HaIxon> John* Xacional Aaiaaaaenc and Che Teaching of Engliah ^ 
NCTEt Urbana, IlllnoiSi 1973. 

Coafflenta i The ucilicy of chis reporr will dapand upon che reader*s arcicude coward 
cha subjecc of reading race icsalf. Thosa who chink ic a crivial subjecc 
will noc find ir useful: chosu vho ara incerasced in rate will find 
liccle choc is new or ut-ex?ecced. :;A£F found few "super readers'' 
(> 730 Wip^m^.)* a fccr which mighc be dismaying co Evelyn U'ood scudenca* 
The major ucilicy of che book liaa in cha face chac ic is a nacicnal 
acudy ol reading race and ic concaina daca on 9'year'*oldSt 13*year*oldat 
l7*year*olds and adulcs. 
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nt of SduCaCiOKAl Prosr«as Vrttlag Objectives > Secon d 
ERIC 11^ ED 072 460» 33 pp. 



' Report : 0^ 



veA Booklet 



^ Writing Aesftifmint 



-3. 17. 



Conte 



:roductloa to HA£F 



- iscrlptlQii of proctdures used to revise objectlvee 
writing Objectives: 

1. Demoostrttee ebUlty In vrlClng to revttl psreoneX feelings 

and Ideas. 

A. Through free expression 

B, Through the use of convutlonaL oodtts of dlecoursft 

U, Demons t rates ehUlty to vrlte in response to t wide range of 
societal dewds and obllgstlons. Ability Is defined to 
include correctness in usage, punctuation, •P«lllngt ano fona 
or convention as appropriate to particular vrlting tasks* ••g*? 
manuscrtptSf letters, 

A, Social 

1, Personal 

2, Organizational 

3, CoBBiunlty 

B, Business /Vocational 

C, Scholastic 

III, Indicates the importance attached to vrltihff sklUSi 

A, Recognlsss ths necessity of vrlting for a varlsty of 
needs (aa In 1 and II) 

B, Vrltes to fulfill those nscds 

C, Cets satisfaction, even enjoyment, froo having vrlttsn 



Related Materials ; All second vrlting easesanjTit reports 
Comments; The differences betvaen these and ths previous objectives are In* 



atnictlve. Most striking Is the first objective, vhlch did not appear 
In the earlier set, the notions that vrlting begins vlth ths satis- 
faction of personal needs and that expressiveness ia critical to 
vrlting are ideas that have acquired reneved Isportsnce in the seventies. 
These objectives are ouch core detailed than che earlier onea, also. 
Each objective la explained at length aod adequacy of expreaslon la 
defined with sample papers. The reader of this booklet coaes away with 
a philosophy of writing^ a feel for adequacy and e fraoevork as useful 
fbr Instruction as for aaaesstsent. Because they are so comprehenalve, 
these objectives are uaeful corroctlvea for people under presaura to 
test •*ba5ic** writing. 
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Ticle : Kaclowa Asaaiscenc of Educational Progress UriEAsg Obj*flCivM. Ig6g» 
ERIC ED 033 370, 19 pp. 



Tvttft of lUporc i ObjCfiClves Sooklec 



Aaa«Bttittntfg> t 1969-70 Writing Assessaenc 



Ag« Lavlftt> i 9, 13, 17, AduXc 

Contont< i - IntroduflClon co SAEP by Ralph Tyl^ar 

- Dticrlpcion of procedures used CO develop vrltiog obJ«flClv« 

- WrlClog ObJecciv«5 

I, Vrlce CO ComtBunlcaCe Adequately in a Social StltcClon 
II, Wrlce CO Coootunicace Adequacely in a Buaineas or Vocacloncl 
Slcuaclon 

1X1. Wrlce CO Commtmic^ce Adequacely in a SchoiaaClc Slctiacloo 
IV, Appreciace Che Value of UriC:Uig 

- t1aa«A o£ individuals responsible for obiftficlvia. 



{(plated ttecarlal i All Wricing Assessmenc reporcs 

C amaats t These objecclves, developed by Educacional tesCiag Servtce lo 1965* 

reflect e "sicuacional" view of wrlcing. Although the word "edequately ' 
la used throughout. It la never defined; the drafters of the objectlvea 
express hope that this probXea can be passed on to readsrs of the re- 
ports, who will Judge adequacy by reading sample esseys. In thair 
refusal to tie graimar to writing and la their leanings toward rhetorical 
theory, these objectives were relatively sophisticated for 1965, The 
absence oC an objective dealing with personal writing, however, was a 
glaring oversight. Fortunately, that would be corrected In che next 
objectives. 
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Tttlt t KatiOHftl AssQSsmtrt of Educational Progress Residing Objectives, 1970 , 
EKIC ^ ED 041 010* 3A pp. 



Typg of Rtfort t Objtcttvea Booklet 



^atM^QtjU); 1970*71 R«adlag Assessment 



Aet Ltvalfs) ; 



13* 17, Adult (26-35) 



CoQttnts i - iredufifiioQ to KAEP 

. ^fi?lpCloa of procedures used to develop objectives 
- Rudiiig objectives 



I. Comprehend Wliat Is Read 

A. Re^d individual words 

B. Reed phrases* clauses and sentences 

C. Read paragraphs* passages and longer words 

II < Anilyse What In Read 

A, Be Able to trace sequences 

B< Pereelve the structure and organization o£ the work 

C, See the techniques by which the author has created his effects 

III< U» Whet Is Read 

A< Remember significant parts of what Is read« 

B, Follow written directions 

C< Obceln Information efficiently 

IVt ReeeoQ Logically from What Is Read 

A« Drev appropriate Inferences from the material that Is read 

end **read between the lines" where necessary 
B, Arrive at a general principle after examining a series of 
deeelXs 

C< ReeeoQ from a general principle to specific Instances 

V< Make Judgpients Concerning What Is Read 

A« Relate what is read to things other than the specific 

material being read 
B> find end use appropriate criteria In making Judgments about 
vhec Is read 

C* Hike Judgments about a work on the basis of what is found 
la Che work itself 

VX« Have eCCltudes about and an interest In reading 

Neaes of individuals responsible for objectives 



Related ^taterlal i All reading ansessment reports 

SaaeaSi* Developed In 1965 by both Science Research Associates and Educational 



TesClQg Service for the National Assessment, these first Heading 
objettlvoe sre very comprehensive* Surprisingly, oany of them remain 
relevant and anticipate the third set of combtned Reading and Literature 
objectlvtis developed in 1978, Although chey reflect a 60's concern 
with aufih things as decoding, word attack skills and speed reading; 
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alchough chey as«uB« a no longer fashionable **boccoa-'Up'\ hierarchicftl 
processine codtl; 4nd fllchough chey aaka discinccions hecwttti lictral 
and itiferericitl cOEpr«h«n*lon which few p«o?l« vouid tupport coday, 
they aeverchelcis proaocft tuch gotls «s: usint conccxc for word 
identificacloa; uodtracaadiQg Jcrgon; coaprehtading scruccurti and coitc; 
incer^reclng figuradva lansu«ge in all ainncr of diacourst; under- 
scaodlng various rhacorical devlcaa; an&lyilng licarary and cxpoaicory 
works; recognizing propaganda; maldng vaXua Jud^nca; and sore* 
The^ are naz parochial* ccofttacic or crivial, as ao aany reading ob*- 
ieccives are. The aophiacicacad vlu find sooa phiXoaophical fussinaaa 
and some placing of aubakilXs under cha wrong inajor skUXa> but cha 
generaX reader vUl find in these «n incereaclng acercing poinc for 
curricuXar or asacaaoanc planning. 
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,T^cle: IfACiOD^ A^sacsQenc of Cducaclonsl Progress Report 5i t/ritingt Croup 
^tsulEs A , ERIC ^ ED 051 246, April, 1971, U3 pp. 

J ype of RtPorc : TdchcUcal Sumcty for T«chQiCally--Ori«iiCed Audi«Qce 

AiB^sfim&titfs) ; 1969-70 Wrlcins AsaessseQC 

Age Levelff) i 9, 13, 17, Adult (26-35) 

lab les t Sex, Region, size of Coamnlcy 

Ipeo Typefal t ^Iciple choice, shore answeri yes/no 

Ds£a; Percentages of success end scaad^rd errors for e&ch released exercise, 
each veriable group, each *ge level. 

DiscrlbuClon of differeaces and aedlAQ difference from cbe oeclonal 
perfonance Isvsli each vsriable group. 

Other Coatent J XncroductloQ CO KA£P; copies of released exercises f sample asesya 
the cop, Che alddle and the botcog of the (holiscically 
judged) quality discribucion; disc ussion ot exercises upon 
which groups perfonoed acypicaXly veil or poorly. 

Source M^Esrlal Aveileble to Reteerchers : Reporc 10 cooCalns e sutple of the 
essays vriccett for this assees&etic 

Ralatad Material ; 1969 Writing Qblectlves , 

Hellcn, Johflj Xsjlonar^sgefsroenE and the Teaching of 
English , NCTE, Urbaru, IlUnois, 1973* 

CoBPents ; See eotr? for Report 3 for brief discussion of problens in reporting 
results of holistic scorlTig* Thasa prableme ere ameliorated sove* 
vhat vheo one is comparing groupa to Che national level of performance* 
This Is so because even If Che national percentage has no definite 
meaning, departures from It by different groups of people do! Thle 
report still cannot tell us hov veil people write, but et least it 
can tell us which groups pecforo better and which voree than average. 

This is a very difficult report to read, pertly becauee the <iata 
have been overanalysed (scatlsCically) , and underthought about* But 
it is worth wading through because it docuiaente unequel achieveaent 
of varioue groups of people. Including adults. 
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Title : NaClonal AsstffsafnC ot Educational Profrea§ Rttport 3s Writing, Natloiml 
Resultf > ERIC & EO 051 2^5, N'oveiaber, 1970* 162 pp. 



Type of Report a XACitiaal Results for General Auditocft 
AaseasmentCs) : 1969*70 tfriCiiig AssessmeaC 
Age Level (s> : 9, 13» 17( Adult (26-35) 



Variables : USqm 

Item Typefs) : Kulciplt choice* cssay/letter» short acisvtr, ycs/no 

Data ; Fercentagai of aucteaa by e^cerclse for uch objactivav each age; per- 
centages of yas/nQ responses; results cosparad for axarclses taken by 
tvo or mora sgs lavals; special study of l3-yaar*o Ids' and L7-year-olds ' 
perfomanca on an assay task assigned to both. 

Other Content : Backgtound on XAEP; writing objactivaat description of data 

gatharins nethods; copies of actual axarcisas used in assessment; 
scoring guidos for short answer and aaaay questions; examples 
of assays at the top* the middle and the bottom of the (holistically 
judged) quality distribution; description of ?rA£P variable groups; 
dasctiption of sampling and weighting procedures; description of 
essay scoring pn>cedures and special overlap study; number of 
respondente by package and age. 

Source Material Available to Researchers : Report 10 coataina a sa&ple of the 
essays vrittan for thii assessment. 

R«l^ted Katerial: 1»69 Writing Oblectlves, 

Heilon, John, National Asseesment and the Teaching of 
English , NCIE, Urbana, lUinois^ 1975. 

CdflBnenrs : with the writing of this first report* it becSM clear that the 

assesement of vritlng was not structured in e vey that facilitated 
conelueions ebout the quality of writing In Aaerice. Each objective 
(Write to COBQsunicate Adequately in e Social Situation^ a Vocational 
Situation end e Scholastic Situation^ Appreciete the Value of Writing) 
uaa uusui vith escerclses which were too fev in mimber and too dif* 
ferent in kind to allow any aggregation of them into summary data or 
general statemenCe. The results (vhich ranged from very low to very 
high percentages) must be considered on en exerciie-by^exercise basie. 



Reporting of essey results founders upon the Achilles heel of holistic 
scoring. Since there are no criteria for excellence and the scoring is 
entirely reletive, all we knov is thet the essays fell into a normsl 
dlsttibution; soma are better than otherSp but ve do not know why and 
the teport cannot tell us why. An ingeniue attompt to solve this 
problem by pteaenting essays and telling the reeder that ''13;: of 
the eseeye vere this good or better" only plecce the burden of criterie 
upon th« reader end leaves us wondering obout the exact nature of these 
unimaginable *'aquelly good or better*' essays. This serious veaknesa 
led to the development of primary^trait scorings uhich enables one to 
describe esaeye in terms of absolute quelity end make concrete Judgments 
ebout improvement or decline. 

Because it conteine data about adult vriting perfomance, because many 
of the writing tosks are interesting in themselves and because ir 
illustrates io well the problems involved in reporting the results of 
holistic scoring, the report Is well worth examining. 
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Title: Second Assessment Lltaraftirt Objectives ^ 1973; IKIC 5 ED 113 737^ 12 pp. 



Type of Report : Objectlvtte Bookltt 

AssessaeotCs) : Because tht atcond llteraturt isstessent wai delaytd by flvt 

years, thest objactlvtti per te htv« ntvtr bten isatssed, P^rts 
of thlff docuntnt wert blendtd vtth Reading objectives for the 
1980-81 Pleading/Literature asaeaaisent. 

Age Level(s) : 9» 13» 17 

Contents : Description of procedures for developing reviaed objectives* 
Background raci(;nale for objectives. 
Objectives: 

I, Experiences literature — ie aware that literary qualities 

exist in a variety of fomSi Seeks experiences with litersture 
in any £om» from any culture, 
A, Listens to literature 

K«eds litereture. 
C« tfitnessea literatursi 
II, Responds to literature — responds to literature in any fotut 
from any culture^ in a variecy of ways — eiootionally) 
reflectively, creatively — and sharea reaponsee with others, 

A, Responds eootionally — participatee enotlonally in the 
world of a work of litereture 

B, Responds reflectively — understanda a work of literature 
by'ref lacting upon it in a variety of ways 

C, Responds creatively — ueea language Ifflsginatively in 
responae to a work of literature 

Sharea responses with othera — shares emotional, 
^flective and creetive responaea in a variety of ways 
III, Valu. "lerature — recognises thet literature plays a 

significant continuing role in the experience of the Individual 
and society 

A, Kecognises that litereture My be a source of enjoyment 

B, Recognizes thet experience vich literature may be a meane 
of developing self-unders tending end personal values 

C, Recognises that experience with litereture may be a aeana 
of underetendlng the neture of man and the diversity of 
culture 

D; Recognizes that literature and society may influence 
each other 

Kecognises rhet literature uy be a eignificent means 
of tranemitting and sustaining the values of a culture 
People associated with the development snd review of objectives, 

Cdopeots: The most striking differences between these and the first objectives 
is the shift in the definition of litereture away from the concept 
that it is a corpus of works, toverd the notion that it Is a quality 
of language use. The stetement of assumptions aays quite plainly 
"Literature is language used Imaginatively/* It follows from this 
radical shift that literary instruction is instruction about particular 
uses of language, not instruction centered upon a liat of "great 
books*' or **litereture of excellence*** Indeed, the goal that people 
leatn about a vide range of clesaio texts disappeared entirely in 
these objectives. The? presume that ell people have eccess to 
literary experience*' as long as the>> heve acceee to lenguege and 
liaagination. From the elitism of the first objectives we swing to en 
opposite extreme of **egaliterian^* goals — owre vegue» to be eure, 
less confident — but more interesting and challenging in coopenietijn* 
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k lack ot coafldtncc In XitatAtute as a dcfiMbXe descipUne wlch a 
vpcclflc conccac tcvMla Icaclf mXso la che staceneacs about valuing, 
Ti^ 1965 ohjactlvaa acccc boldly '^Recogniit that participacinS In 
tbm ilccnry txpcrlCQCc it a prima form of eajoynanc**; In I975i 
hovarati cooaulccaca fait wr« comfortabXa wich '*Racognlza« thac 
LlcaratUra nav ba a aourta ot anjoyokenc" (Emphasis oiina). Soma 
would vim chit nodlflcaclOQ aa a loag-overdua dascanc i!rom cha ivory 
covar; othars will aaa It aa aba&doniDftnt of beliaf or aubmddsioa to 
tha aacl-lACallaetualin ot cha 60s and aarly 70a. 

Ilia 1965 and 1973 HAEP llcaracura objactivas rafXacc coasidarabXc 
ebangaa la atclcudaa coward llcaxiry instruccioa afcar a profound 
daeada of Aaarlcaa axparlanca, liay ara culturalt aa well as 
aduaacloiul documaaca* vorch raflecclng upoo for culcural* aa well 
aa aducacloDf raaaooa* 
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Title s wrtt*/R^glttt An Aaatflsmgnc of Rftvlalon Skill* , ^TftClonal Assatsscnt o£ 
* EductClonal Progrfits RtporC a5-;f^0A, July» 1977, 30 pp, 

Typ» of R<Port i Ssltcctd lUiulcs for Central Audlenct 

Agaessmctatifti i 1969*70, 1973«74 Pricing AssessoenCs 

Age LeveKgl i 9p 15p 17 

Varlables f RigloUp S«x» RAct* ConmunlCy Type* Farencal Education 
Item Type<8) t Eaaayp laccar. 

Bata: Nacional and varlabla gioup percencagaa of papars ac 4 lavals o£ quality, 
agaa 9p 13 

Parcancagaa of paopla ravlslng chair papers/XeCCari 

Paroancagaa oaklng cha following kinds o£ revldlonai coaaacicp ntchanlcal, 
granacicalf concinuaclonalp InformaClonal, Cransicionalt acyllacic, 
orga&lxacional, hollacic 

Other COBttnt f Explanaciona of cams, scoring procedtiraa, ace. 

Source Macarial Avallahla to ftaaearcfaera : CooCacC NABP 

RelaCcd Macarial i tfrlClna Tae^^fflcal RePorC: Released Eacarelaa Vblona 

Coomeocy : Tha flaws ia chla acudy should uoc decracc from cha face chac ic is 



an axcraordlnary effort Co gather naClonal daca on a crlclcal wpecC 
of' cha vrlclng process. Nine* and 13-year-^olds vara given five 
facca abouC cha moon and were asked Co organise chea mco a P*P*r 
abouC cha moon. The reporc describes Che organleacional quelicy o£ 
Che aaaays and chen describes vhac happened when Che acudenca were 
.cold CO **revl8e** chelr papers. SevanCeen-year*olda vere given a 
rhecorlcal alCuaClon Involving a leccer Co a negllgonc grocer; Chelr 
leccera vere evaluaced in terss o£ chelr concenc and approprlacaness 
of cone. They ware elso subjecCed Co Che sane revision analysis as 
cha youoger scudenca* papers were. 

One can wish for more; for an asslgiunenc chac covered til chree 
agaa; for aore rigid "before" and "a£cer" qualicy avaluaciona; for 
varloua ayacaccict or llnguisClc analyses — and so on. 0uC chcre Is 
plency of uaaful InfomaClon packed InCo chls llccie rsporC and IC 
represencs a productive sCarC In chls area of reeaercH. The cace- 
gorles of revision enalysls can be toore clearly defined by fuCure 
researchers and clad more meciculously to various llnguiscic and 
aenanclc levels; buC even as chey sCand, Chey have obvious classrooa 
Iftp Ilea Clone. 
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Ticlq : WatlOMl Assen$mur\t R&PoTtJJ* Writing , ERIC 9 ED 077 029* 1972* SO pp. 



' ^pf y( TtcPort i Seleccad Results for General Audlenct 
A ffMwentdl i 1969-70 VtlClng AiStssoeaC 
Aftt Lftyglfai i 9, 13, 17, AdulC <2(^ZS) 

Virltblea l Riglon* $m* Color* Slxt and Typt of ComiDunlcyi Partncal EducaClon 
Itm Typefsl : Mulciplt choice* tssAy/leCCttr* shore ansvtr 

StSi} Perctncages o£ aucce^a for aach age group and variable group wichlA aga« 
PteformaiSce cooparlaooa on leans cooaon Co two or ooore ^gta. 

Ochar ConCi.>pc t Exerciaaa. ^ 

DlscuaalOQ of rasttlca. 

Sourca Material Avatlabla co Reaaarchers i ConcacC NAE7 

Halatad Matatial : Helloo* John* National AaaaasoenC and cha Teachlng_of 
Entllah . HQTl* Urbana, Illinois, 1975. 
tfrltlnfl Qblgcclvaa , 
Rgport 5 * 

CoMBflitta ; Thla report vaa an afterthought attatnpt to praaent flrat aasesamant 
vriting reeulta la yet another focn. Report 5 had praaantad suaufy 
raaults for all egea by aax, region and size of comounlty* Thla 
report addraaaea the additional variables of race* size end type of 
eonunity and paret^tal aducatloo* but It pteaents exerelae* level 
reaulta, not (as Report 3 did) aunary reaulta acroaa ell exereiaaa. 
It d^cuaienta differential perfomanee by -various groups of people at 
different egea on quite different teglcsp ao it la worth perusing. 
Although it la organized by objecCivei generalization from the 
reaulta of two or thraa tasks to eoncluslona abuut achievemeitt of 
evezy objective renaioa impossible. The queations and results related 
to Objective IV» '^Appreeiete the Value of Writing/* are quite interest- 
ing and do not appear anywhere alae* 
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tltltt i UndTatandlntt laattinfltlv^ Languaf^e , llatloiial AssessmenC ^lepo^C C2*L*0U 
ERLC CD 077 002, 1973* 197 pp. 



yy^m of RBDort i S«liccaci RiiulCd for teneraX Audience 
/l^rtMg<ntfa) i 1970-71 Llcat«Cur« AssessioenC 
Aat Uvlfi) ! 9, 13, 17, Adult (26-35) 

VjjE^^Afi^' Kiglon, StXt S12a and Type of Comnunlty. Race* parental t^uetcion* 
Slat of Comunlcy 

Itm Typafil l Hulclpit eholct, muXcipIe choice wlch n^ioit ansver d«fQn«i of 
cholcOt 

Pata i ?trctRC«gift of 9-y«ar-oldst l^year^oldsi , n^-year^olds ^cd adulCfl atlicclRg 
corttcc acavtta* 
Dlffittncta from naclonal percenCagea for variable groups^ 
SUodard ttrora for §11 perceaCages, 

Dlactlbutlona of group dlfferencea iot all exercises la report* 

HadlaR dlfftttnct from che nation oa aXX exercises for «ach vatiablt group* 

CoapatliORS of 9, 13, 17 and adult performance on connnon iteos* 

Other Coutant i Scoring ctlcttia for shorC^ansver respooses. 
Sample reapoaaes. 

Source Hatarlal Available to Rasearchets : Raw results on 1600 SPX* 9-rtack 
data tapaa usable only on ^BM 360 or 370 hardware with <>r V3 operating 
ayatana 9 $94t00/tapat 

Halitad Material i 02-GIYi Reeding aad Literature Cenera:^ Itkfottpatlon Yearboo^ . 
Lltiratura Objectives, First Asaessroent . 
Hallon, John* National Assessment and ghe Teachlna of 

Engl^ ,^ht EfCTEr Urbana, Lllinols* 1973. 
GrlRdareff, ?aye Louise. '*Th« Macional Asses5ii»^nt of 

Literature] A Review," Research In the Teaching of 

ERgllah . 9j1tS0'97» Spring. 1975* 

Comenta: This report deals ulth thode assesaiment exercises designed to probe 
UBderacaRdlRg o^ poetic rhythm and logic, pun^, metaphors eud genres 
as well ea thoso aaffesalag liiference in poetry. These areea do not 
by any meens exhaust the field of "Inaglciative language,** and herelR 
ll«f a butXt-'ln lloltatlon of che report* Much was not assessed end 
sou of what vaa aasessed was not done well or thoroughly* SlRce 
results ere reported exercise by exercise* the reader can pick and 
choossi but cdAAo^ generalize with much confidence* 
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